What are the challenges of automating a fiber carbon winding automation?

Depends on your product requirements, but in general a CF "winding" setup required multiple degrees of freedom -- the target mandrel usually has to be on a multi-axis positioner of its own, in addition to the filament tool's own degrees of freedom. CFW needs to make many layers, wound at different angles, so that every layer has a different "grain" than the the ones below and above it.

Then there's the issue of maintaining proper tension on the filament while doing all this reorientation, all without catching the filament on anything, or getting it tangled. And if you're laying a "ribbon," rather than a string, that adds one more dimensional constraint.

Then there's potential issues with dry vs wet wrapping, solvents, adhesives, potential toxic and/or explosive vapors, sometimes a need for a strictly controlled environment (temperature, atmosphere, etc).

And to top it all off, the software challenge of turning a winding pattern into a comprehensible multi-axis program for the various positioners, and creating the kinematics for the machine... it makes creating 5-axis CNC software look simple. There's a reason this tends to be a very specialized field, with not many players.