SRVO-068 Continued...

    I believe our engineer is just as curious as the rest of us (only 4 of us in my department) but another issue we have is overtime has been completely cut for the time being. I'm hoping that here soon the overtime will come back and we will be able to come in on a weekend and pull the other three cables off the robot to start testing. I know the cable from ARP2 to the 7th axis encoder is ok because we've been through at least 9-10 of them probably and chances are slim that they were all bad. We actually kept three of them on the robot and would swap back and forth between them each time the alarm would occur. The cable from ARP1 to the controller should be ok as it just lays in a cable tray but I wouldn't rule it out completely (we ran a new one of these too). I do know that we will be holding a couple of these new one piece cables on hand for future occurrences so that's a plus!

    "I could tell that my parents hated me. My bath toys were a toaster and a radio."

    Hey guys, me again. :wallbash:


    So after installing the one piece cable from the servo amp to the encoder we rid ourselves of this dreaded DTERR alarm for about 2 months or so and assumed it had to have been the internal harness. Well guess what has come back to bite us yet again? Luckily we had ordered more of these one piece cables so our third shift guy ran a brand new cable back out from the amp to the encoder and this made our DTERR fault occurrences DOUBLE to almost 4-5 in two hours! The cable was ran correctly and grounded and blah blah blah. Meanwhile 4 other robots are doing the exact same job and are set in the exact same environments and we have zero issues out of them. What in the world could possibly be killing these cables? I threw another encoder cap at it a few days ago and we had zero change in the amount of DTERR's we have a night. The cable itself is wrapped in leather for added protection and even wired to its own battery box inside the cabinet.


    I would hate to think that the cables are so fragile that it was damaged during the install process. It's not like the guy was swinging from it and dragging it across razor blades :uglyhammer2:


    The only reason i revived this thread instead of making a new one is because of all the info I've already typed out is in here.


    Another question, is there any way to use this servo as a simple ON/OFF without needing to worry about the encoder data? I know they make motors like that but it would be way to costly for us to switch to this method this late in the game since these lines won't be around but another year or so. Also, just to reiterate, I have an identical controller and robot that I can pull and swap whatever pieces I need from so give me some ideas please!

    "I could tell that my parents hated me. My bath toys were a toaster and a radio."

    Edited once, last by SHIFT_Lock ().

    Do you know categorically whether it is the cable that is damaged?


    I'm just throwing this out there, could this problem actually be down to the retaining connectors/pins/sockets (external to the cable) creating intermittent contacts.


    I would certainly be inspecting these closer to see if they are corroded, ingress with oil/grease, pushed in slightly, wiggling inside the receptacle, dry joints etc.


    I'm not sure about the 'servo as a simple ON/OFF' reference......sorry.

    We aren't %100 sure it's the cable but that cable is what resolved our issues for over two months. As far as the connectors and pins ect. the encoder cap is new, and the external axis amp was swapped with the other robot. These cables bypass the internal harness entirely and it's a direct unbroken run from amplifier to encoder so the plugs on the robot do nothing at this point. I almost feel as though something is intermittent and possibly "shorting" out these cables. Although i'm at a loss with why the brand new cable doubled our occurrences of this alarm. Besides the external axis amp i'm unsure of what else could be swapped inside the controller cabinet to narrow this down but i'm open to ideas as i have an identical cabinet to swap parts between. Only thing would be getting the time when they aren't running production. Overtime is pretty much locked out currently.

    "I could tell that my parents hated me. My bath toys were a toaster and a radio."

    Just reviewing the entire thread, there's been what I would refer to as a ridiculous amount of exchanging of parts (I don't mean that in a derogatory way).

    In times like these, I often review everything again as the whole experience has just saturated all apparent logical routes of resolution and wondering if it's worth looking at this from an alternative direction.


    I have my own suspicions of this stoppage error being the result of a problem and not necessarily the cause.


    I know you have minimal opportunity to do anything proactively with this and I don't wish to send you down a completely unrelated road but...….


    1. At the point of failure, what position is the physical posture of the manipulator in (axis 1 - 7).

    2. Can you tabulate the positions of axis 1 - 7 at the point of failure.

    3. Have you an infra red thermometer you can shoot at ALL the encoders and tabulate them at the point of failure.

    4. Are there any other errors associated with this at the point of occurrence.

    5. I would be very tempted (not that I would recommend) to force this error by disconnecting the pulse cable from the external axis whilst it's running and see if the resulting error is the same.

    I completely agree with you that we have been playing parts roulette with this robot. I think some of it is due to us just not knowing what to do next and the fact that we have identical robots to interchange parts with at no cost to us. Not to mention the sheer frustration we've all been through.


    1. At the point of failure, what position is the physical posture of the manipulator in (axis 1 - 7).


    I actually had a DTERR fault while typing this and this is what I got.


    J1: -53.521 J2: -.359 J3:-32.809

    J4: .000 J5: -57.187 J6: -157.068


    J2/J3 Interaction: -33.169


    G2 J1: 170.556


    This is typically the same general area where our DTERR fault occurs although it has happened in a couple other positions as well


    2. Can you tabulate the positions of axis 1 - 7 at the point of failure.


    I will begin to collect this data at the failure points throughout the night to check for any consistencies.


    3. Have you an infra red thermometer you can shoot at ALL the encoders and tabulate them at the point of failure.


    I believe we do somewhere in the plant and I will talk with our facilities guys to see about getting it. As a last stitch effort I do have a really good infant thermometer in my truck that is designed to check in a no contact manner that I could try haha.


    4. Are there any other errors associated with this at the point of occurrence.


    This is the only error we receive during this failure. Every failure we disconnect the encoder and re-calibrate the extended axis. It's a continuous turn so it only takes a couple minutes.


    5. I would be very tempted (not that I would recommend) to force this error by disconnecting the pulse cable from the external axis whilst it's running and see if the resulting error is the same.


    With this it would be hard to do "safely". I imagine I could put the line in teach and rotate the extended axis via the TP and then unplug it. Or during normal operation I could probably just unplug the amp side of the cable to achieve the same effect.


    I do appreciate all of your help and interest in this matter. I would still consider myself a novice when it comes to robotics as i've never been to any kind of training and what I do know i've learned by playing with a TP, reading on the forums, or learned from a co-worker.

    "I could tell that my parents hated me. My bath toys were a toaster and a radio."

    Thumbs Up 1

    Yes, that type of data is exactly what I was referring to.

    Regarding the 'I'd be tempted to remove the pulse coder cable'...…..yes completely ignore that, I just mentioned that from a 'Me' perspective.


    It's clear you can recovery, reset and continue with production, which rules out (IMHO) circuit board failure within the Controller.

    Therefore, you do have some time available to 'study' and 'focus' a little more, in order to try and understand what's going on as opposed to 'reactively' responding and just swapping parts in the hope it cures it.


    I'm hoping this type of approach may begin to yield some 'repeatable' information at the point of failure, where you could highlight:

    1. Specific postures of the robot, not just the Joint the error is relating to.

    2. Program and step no. when the error occurs.

    3. Temperatures at the point of error (Not convinced this is heat related, but the more information you can gather the better).


    To the point, where you could possibly then:

    1. Create a small test program, copying the original 'program snippet' where the failure is occurring.

    2. Running some test cycles (when you do get some time) to see if you can in fact repeat the error.

    3. Thus removing any external processing that could also be influencing this.

    4. Leaving you with a distinctive picture of the intermittent error, now being repeatable in a more isolated and controlled environment.

    J1: -53.521 J2: -.359 J3:-32.809

    J4: .000 J5: -57.187 J6: -157.068


    J2/J3 Interaction: -33.169


    G2 J1: 170.556



    J1: -52.717 J2: 5.614 J3: -32.251

    J4: -.000 J5: -57.758 J6: 27.306


    J2/J3 Interaction: -26.636


    G2: Forgot to document


    J1: -53.366 J2: -.451 J3: -32.805

    J4: .000 J5: -57.190 J6: -157.210


    J2/J3 Interaction -33.259


    G2 J1: -68.682


    This is the positional data for the last three occurrences i've had so far. The first and third time have been basically the same spot in the program. The second one was on a different weld with a couple of the joints in a relatively similar position.

    "I could tell that my parents hated me. My bath toys were a toaster and a radio."

    So it does appear to be in very similar areas then...…..interesting...….I'm thinking there's another harness that could be influencing this error.


    When you reset the error at the point of stoppage:

    1. Do you recover the robot and repeat the process again, or do you carry on from it's error position?

    2. Also, can you make a note of the Program and Step No. too

    After a DTERR alarm our process is usually to cycle power and return home to restart, or pull the pulse coder cable to re-calibrate and then return home. Pulling the cable seems to make it go longer without a fault so lately I've been doing this every time.


    I will start recording that as well as the positional data.


    Originally I did wonder if it was another axis' cable causing us grief but the new cable is ran entirely outside of the robot currently so I really don't know.

    "I could tell that my parents hated me. My bath toys were a toaster and a radio."

    So you are moving the robot away then and restarting as part of your recovery.


    Any chance you could try just 'carrying on' from where it errored and stopped after 'pulling the cable', does your process allow for this?

    - If so, and it errors straight away, it would suggest you are now in a permanent error condition.

    - Then you could start pulling on other cables and see if it then gets it away.

    - If this is the case, then you could locate the 'suspect' cable that way...….if it is cable related.

    Yes normally when we get this fault we drive the robot up to clear the jig and remaster. I'm sure we will have another fault before the end of my shift and I will try your method to see what happens. I'm not sure that we are setup to continue after a power loss though which is the only way to clear the alarm. This extended axis also isn't near any other cables and the encoder cable is ran on the outside of the robot away from the bundle. I'm wondering if this could be an issue with the power/brake cable inside the robot? Possibly temporarily shorting and stopping power/communication to the servo? Honestly I think that's the only thing we haven't swapped.

    "I could tell that my parents hated me. My bath toys were a toaster and a radio."

    Thumbs Up 1
    Quote from SHIFT_Lock

    I'm wondering if this could be an issue with the power/brake cable inside the robot? Possibly temporarily shorting and stopping power/communication to the servo?

    This is exactly where my thinking is too.

    Also, would it be possible to access these (especially where they are clamped and where they are free to move) and have a good inspection/pull on these too whilst it's in the error stopped location.

    Ok, so doing some quick reading it would seem I may need to change a System Configuration setting "USE HOT START" to "TRUE" and I will be able to continue the program after a power recovery. So I will try this next, although I do believe i'll have to set it and cycle power first for it to be "enabled".


    *EDIT* Scratch that, they are already setup to start "hot" so I will need to do a little more reading on that subject. I remember seeing something about START for CONTINUE only or something along those lines.


    As for the cables, yes I can get to most of them fairly easily so if this HOT START idea works like I think it should I will be able to test this out.

    "I could tell that my parents hated me. My bath toys were a toaster and a radio."

    Edited once, last by SHIFT_Lock ().

    Only change a system configuration like that it you're sure you have no other processes controlling this like BG Logic or PMC.

    Otherwise, (lower risk), give all those cables a good inspection and yank whilst it's in error at the location it stopped in.

    Then move it away like you normally do and restart.

    The problem with this, if it is those cables, it may error out straight away, which means your production is stopped, but in earnest you'll have a better idea of the cause if this is the case.

    I'm reading into it more and I don't think there's any reason I couldn't make the change provided I find out exactly what it is that needs changed. We don't use BG Logic or PMC in any of our robots so that's not an issue. We use an Omron PLC and the robot uses STYLE to call the main job. So I don't see why I couldn't have it just send a start signal and carry on. We already have it setup to recover all I/O upon power recovery. I think our restart issue is not with the robot config but with the start signal sent from the PLC. Either way our production is almost over for the night so I will continue this Monday when I have the line to myself to tinker away while they run the other line. I really do appreciate all your help this far.

    "I could tell that my parents hated me. My bath toys were a toaster and a radio."

    Thumbs Up 1

    You're welcome.....


    For the purposes of trying to diagnose this, I wouldn't be too concerned with exploring/making changes in that respect as you never know, you may accidently introduce another problem...…...


    If you can give everything a good inspection and yank, whilst it's in the error location and then carry out your normal restart, and you suddenly notice a change in where the error occurs, then I think it'll be a good enough indication of where the cause is likely to be......if it is the other cabling.


    Hope you get there soon...……….

    If the motor is still running hot, the PulseCoder may be the issue. The constant high heat causes some to be more susceptible to noise after a while.

    Have you tried swapping the entire motor to see if the heating issue resolves?

    If it is a noise issue on the data lines as the reply you received from Fanuc stated, I would still be looking at the shielding being grounded at the right spots.

    The servo motor was swapped between another similar robot a while ago and since then we've replaced it again but I can't remember the fault we got that lead us to change it. It was something directly related to the servo being faulty according to the manual. As for the grounding this new cable doesn't have anywhere to ground it besides the bar inside the controller where all other cables ground to. Normally the internal harness would be grounded through the connection points in the arm and the base but since this cable bypasses those it just runs straight to the encoder.


    I did notice something odd today that may or not be of any relevance but it's worth mentioning. I opened up the positional data screen on both robots for G2 J1 and noticed that Robot A is processing the data much much faster than B robot. The data moves so quick on A robot that I can't really catch any specific number but on B robot I can almost read every one. I don't know if this makes sense so I've uploaded a video to show what I mean. Also at "home" position A robot returns this axis to .000 while B robot is usually around -.084. Both reference positions for home are set at 0.000 +/- 1.000 for this axis. So to me it would appear this may be the CPU processing data at a really slow rate? Maybe the communication issue is timing related and not a broken connection or noise? Or possibly the noise is causing this slow down? Either way whether it's important or not I wanted to throw this out there because it seems odd to me. It's also the only robot we have that's processing this data so slowly and it's on all 7 axis as well. I watched both robots multiple times in G1 and G2 and it's the same result.


    **EDIT** the video uploaded in poor quality and upside down so i'm going to send it again in a minute.

    "I could tell that my parents hated me. My bath toys were a toaster and a radio."

    Edited once, last by SHIFT_Lock ().

Advertising from our partners