IRVision - Precision Pick and Place

    Hello guys! I have been struggling with this for a while now, even posting about it here until I've stopped working on it for almost a year. Now this thing is back to haunt me once more....

    With a FANUC LR Mate 200id/14L IRVision 2D Pick and Place application with 2 cameras. One is mounted on the robot for part location and another in a fixed position for tool offset. The issue is 500~600 microns offset in the X-axis. Y and R are pretty acceptable.

    Application overview:

    1. Robot located the part roughly using found position;
    2. Robot moves with the camera where the part was located, centering on its FoV to minimize lens distortion, then runs the same job to locate the part again;
    3. Robot moves with the gripper to the part's position and performs the picking operation;
    4. Robot moves to the UF located on the center of the fixed camera to get the Tool offset, which should be very small (which is for R and Y but not for X).


    Applications Details:

    • All UF were created using AGFS. The UF where the part is located has a scale of 0.04mm/pix with a mean error of 2.1pixel and max 5.58 pixel;
    • The TCP was set using a CAD model and then manually fine-tuned to minimize W and P deviations due to the physical assembly of the gripper;
    • The Vision Data was created using a 2-plane calibration, with a scale 0.045mm/pix, mean error of 0.258 pixel and a maximum of 0.872 pixel. The standoff distance is 120.7mm.
    • The vision job was made using Window Shift with GPM and then Combination Locator with a set of Position Calculations to precisely locate the part geometric center.
    • The part Z height is set on both the picking UF and also for the Tool Offset.
    • The part's weight is negligible, very light.
    • The robot looks pretty much aligned and has not been into serious crashes. I ran Vision Master Calibration last year and it did make a difference on the results.

    The Pick Operation:

    I basically use 3 PRs to perform the picking operation. I store the Found Position coordinates in PR[99]; I have a PR[3] set to 0,0,0,180,0,0 NUT 000 and another one just for the height offset approach which is PR[5] set to 0,0,-50,0,0,0 NUT 000.

    I perform a J PR[3] FINE + Offset PR[99] + Tool Offset PR[5]

    Followed by a L PR[3] FINE + Offset PR[99].

    Once I am in position I wait 500ms and turn the vacuum on, wait 1 second and then I do:

    L PR[3] FINE + Offset PR[99] + Tool Offset PR[5]

    Then I move to the fixed camera and run the Tool Offset job.

    The idea behind the application is that I locate the part's feature, and move the TCP to that coordinate so I can pick it up. Then once the part is picked I move the robot to the fixed camera UF which is centered with the fixed camera. At that stage, the Robot's TCP, the part's feature and the fixed camera UF are all aligned, with minimal error, which matter of fact is for Y and R ( between 60 to 110 microns offset for Y), but for some reason that I can't figure it out the X is way much higher, ranging between 550~600 microns and it is quite consistent, so it is not some pickup error due to the part jumping on the gripper or something (which it isn't ).


    Anyways, if anyone has ever dealt with this kind of problem, please help me out here.:tired_face:


    Thank you!

    New information: The offset error is consistent with the X-axis of the UF. So, if the part is resting at 0/180 in the tray and then I pick up, the Tool Offset will be at the X axis, however, if I rotate the part at 90/-90 in the tray and then pick up, the Tool Offset will now be at the Y axis of the tool.


    I've already tried to recalibrate the Camera Data and also run AGFS again but the problem persists.

    Edited once, last by AlexMac (August 28, 2024 at 12:36 AM).

    500 microns = 0.020 inches.

    Articulated arm robots don't claim to be that accurate. Repeatable, yes, but they generally won't achieve accuracy better than that for dynamic applications.


    There are some robots that can achieve higher accuracy with calibration, but I don't believe that's on option on the LR Mates.

    You may be able to use the vision adjustment command to tweak it in.

    Quote from HawkME

    500 microns = 0.020 inches.

    Articulated arm robots don't claim to be that accurate. Repeatable, yes, but they generally won't achieve accuracy better than that for dynamic applications.


    There are some robots that can achieve higher accuracy with calibration, but I don't believe that's on option on the LR Mates.

    You may be able to use the vision adjustment command to tweak it in.

    It's possible, but you have to iterate in. I just got off a job where we were doing +/-0.005", but it took about 12 iterations to get there.

    We also used vision offsets to do it. The job was very similar to AlexMac's process. Pick part with vision, go to stationary camera, find where we picked up the part (iteratively), store that tool offset in a PR, go to the place position, find it with vision (again iteratively), and place with the iRvision offset, and the tool offset applied to the place position.

    Check out the Fanuc position converter I wrote here! Now open source!

    Check out my example Fanuc Ethernet/IP Explicit Messaging program here!

    Quote from gpunkt

    Is your reference position/pick position in the vision process set to 0,0,0,0,0,0 in the offset frame used?

    Why are you not using vision offset?

    Hello! Thanks for your input! The Pick position is set to 0,0,0,180,0,0 with the Offset set to the Found Position on the Vision Job.

    I am not using Vision Offset because I find the part using Found Position, not Fixed Frame Offset.

    Quote from HawkME

    500 microns = 0.020 inches.

    Articulated arm robots don't claim to be that accurate. Repeatable, yes, but they generally won't achieve accuracy better than that for dynamic applications.


    There are some robots that can achieve higher accuracy with calibration, but I don't believe that's on option on the LR Mates.

    You may be able to use the vision adjustment command to tweak it in.

    What bugs me is that is accurate enough in one axis and the other one is not, so I think I might be missing something or something is wrong with the robot....

    Quote from Nation

    It's possible, but you have to iterate in. I just got off a job where we were doing +/-0.005", but it took about 12 iterations to get there.

    We also used vision offsets to do it. The job was very similar to AlexMac's process. Pick part with vision, go to stationary camera, find where we picked up the part (iteratively), store that tool offset in a PR, go to the place position, find it with vision (again iteratively), and place with the iRvision offset, and the tool offset applied to the place position.

    That's awesome, Nation! Were you using Found Position as well to find the part? How were you adjusting the pick position? Regarding the parts to be picked, were they in a fixed position or could they vary in position/orientation?

    Quote from AlexMac

    That's awesome, Nation! Were you using Found Position as well to find the part? How were you adjusting the pick position? Regarding the parts to be picked, were they in a fixed position or could they vary in position/orientation?

    Pick position was adjusted using a fixed frame offset. We used the vision offset method. No iteration on the pick. Parts could vary in X, Y, and Rz.

    Once the part was picked, we used the stationary camera, and a tool offset vision process to find where the part landed on the gripper. Iterated here until the error was low enough to proceed. Found vision offset was stored into a PR for the drop position.

    Code
    J P[1:Camera Snap] 15% FINE Tool_Offset,PR[75:Vis Tool Offs]    ;
VISION RUN_FIND 'VIS_TOOL_OFFSET'    ;
VISION GET_OFFSET 'VIS_TOOL_OFFSET' VR[9] JMP LBL[999] ;
PR[75]=VR[9].OFFSET ;
!Repeat above until PR[75] stabilizes. ;

    At the place position, we used fixed frame offset again. Iterated the camera shot here as well. Placed using the stored tool offset, and the found offset.

    Code
    L P[1:Drop Pos] 100mm/sec FINE VOFFSET,VR[1] Tool_Offset,PR[75:Vis Tool Offs]  ;

    Also, I forget to mention that this was done using a LR Mate 200iD/4s. Those are pretty tight coming out of the factory.

    At no point did I use the found position.

    Check out the Fanuc position converter I wrote here! Now open source!

    Check out my example Fanuc Ethernet/IP Explicit Messaging program here!

Advertising from our partners