With a spherical TCP and a stationary sphere, wouldn't you have to determine the center of the stationary sphere first with a pointy tool, then do the whole thing again with the TCP sphere?
This is where it's tempting to break out a laser tracker, but I assume you don't have one in your back pocket. 
If you have a regular dial indicator, my first thought is to just write best-estimate values into the TCP (maybe 3-point a TCP on a mark on the sphere), then keep touching-off the sphere at 0deg and 180deg rotation around the TCP on the two axes that have 180deg contact available, tweaking the TCP value until the deltas are minimized. Then tweak in the axis that's along the shaft using small rotation angles.
IIRC, there's a thread where we once discussed using the intersection-of-spheres method to work out the center of a spherical TCP by touching it to a fixed pointer at multiple angles, but I've never actually tried it. It should work mathematically.
i was not long ago planning to do something similar.
I found this and figured this would be a good starting point.
I have not gotten around to implement it yet but i planned to measure the ball tip with a micrometer to get the diameter. As the tip in my case was a measuring probe i was aiming to get the points required by using a "fixed tool" or just a flat known surface (calibrated as a base) to touch the probe.
Do you know roughly where the tcp should be?
Could you use a fiber fork sensor and get the X & Y of the tool by passing the tool with diffrent sides against the beam. This assuming the tool the stylus has somekind of shaft that you know the orientation of. if the tool orientation is unknown you would need to do this at atleast 2 diffrent heights.
To get the length could you use a Tool length sensor, even crappy ones usually obtain a precision of 1/100ths of millimeters.
Fork sensor: Buy Fork light barrier SOOF online | Festo FI
Tool length sensor: Sorotec Online-Shop - Tool Length Sensor Sorotec WLS 1 (8 mm)
you can create a machined ring that is just slightly larger than the diameter of the sphere. You can then touch off the inside of the ring and get a good TCP depending on the amount of error you can allow in your process.
user is supposed to just place the reference sphere (datum sphere) within robot reach, jog robot till it is couple of mm above it and run program. then robot would use interrupt to sense contact and from few measurements determine TCP.
Mmm... should be doable. Just spitballing, but if the program started with a search pattern to find the top center of the stationary sphere, then moved off to do a horizontal search from X+, X-, Y+, and Y-, generating the position of the center of the stationary sphere should be possible. From there, it should be possible to extrapolate the TCP of the carried sphere. Obviously, both spheres need to be near-perfect spherically and of known diameter.
you can create a machined ring that is just slightly larger than the diameter of the sphere. You can then touch off the inside of the ring and get a good TCP depending on the amount of error you can allow in your process.
i hear you. in fact the 'ring thing' does not need to me machined. it would be enough to drive hot metal sphere into a piece of thermoplastic or produce cast (plastic for example). then one has really tight fitting block.
regardless...
not sure how would that work without previous knowledge of the
precise location of it. and i would like to avoid need for CMM/laser tracker measurement or similar. i would prefer to make it completely standalone (robot only). sort of like placing a reference pin in any location, then measuring TCP of a tool using XYZ method. the placement of the reference object (pin in this case) was completely arbitrary and its precise location is not known - or needed.
using optical sensor was one of the first ideas. after all this is something that was tried and true, used on previous projects the past to locate things, measure tool etc. but for this case, ultimately decision was made to not use that in this case. we want to use tool (already a sphere on a stem) make a mechanical contact to some reference object or jig during calibration. and having the reference object in arbitrary place would be a bonus.
i suppose one of my questions is what shape of the reference object would be better suited for semiautomatic TCP determination? sphere? three touching spheres? hole? cylinder? square block?
just spitballing but assuming you can detect when the stylus ball is touching something and using a sphere as the reference object.
one could make a function to find the center of a sphere from 4 or more points. If more than 4 points somekind of error minimization or "fitting" of the points would be required.
I imagine if you peck at the reference sphere from one orientation at a time and use $TOOL=$NULLFRAME and $BASE=$NULLFRAME get the flange position for the pecked point in relation to the world coordinate of the sphere.
E.g.
Pecking 4 points from above (world Z) and calculating the sphere center from those points. These points would be floating above the sphere at an unknown height so the Z coordinate of the ref sphere is still unknown.
Then..
Pecking at the ref sphere 4 points in the direction of world X. Calculating the sphere center from these points would yield the correct ref sphere Z but the wrong coordinate in world X
You get the idea. Combining the results will give an accurate coordinate for the ref sphere.
Knowing that neither tool or ref sphere changed during the course of measurements one could make an optimization function that finds (or rather minimizes the error) the tool that would make the above 2 measurements yield the same ref sphere center coordinates. I imagine looping thru the tool x,y,z values at incrementally smaller values to obtain a decent result.
Edit: the pecking must be done with the tool pointing in the pecking direction otherwise the above is useless
regardless...
not sure how would that work without previous knowledge of the
precise location of it. and i would like to avoid need for CMM/laser tracker measurement or similar. i would prefer to make it completely standalone (robot only).
I'm not sure what you're getting at? You can do an XYZ 4-point with any arbitrary addressable point within the robot reach on the pendant. No fancy lasers or feedbcak needed. In this case you are using a ring, or as you mentioned better yet a dome that is the exact imprint of your tool, To act as a sharp point to find the center of your spherical tool.
Place your Ring/hemispherical divot tool on your work surface. perform XYZ-4 point, get the spherical tool to fit snug in the hole, take a point, repeat 4x and check your error on your calibration. You're done. Next time you calibrate you can tape it to the wall, the floor it doesn't matter you're still performing a 4 point calibration the same.
The other way to do this, is if you had a tool holder (I'm thinking like a ball nose end mill or CMM probe) you could make a tool that had the same length but was machined to a sharp point you could touch up there and transfer over or measure the difference between the points and offset/input your TCP. Assuming the tool isn't easily removable or they won't give you the budget to have a custom insert made the ring/divot method would work just fine.
There exist a commercial solution that works like your idea.https://www.ims4robot.com/en/
They use two probes with spherical tips (think that's not necessary for only measuring the TCP, the fixed probe is used for some more things to measure).
For measuring the TCP you have to teach 8 (iirc) starting points with different orientations, where the TCP probe sphere touches the fixed sphere, and for each of that point also a point some mm away from the fixed sphere, I the direction of the probe axis.
Think you really need more than just one position above the fixed sphere, as there are too much unknown parameters in the resulting equations.
