This test went... poorly.
We unloaded the robot under its own power, drove around the pavement a little bit, and overcame a curb with very little trouble. We were just starting to take it up to speed and trying some sharp corners when a series of bad things happened. The joystick plug got tugged loose. There is something you need to realize about the way that joysticks work. They don't tell the computer what the position of the handle is, they just tell it every time it changes. It is up to the computer to figure out where it is at any given time. This means that if the computer stops receiving the signal, it just assumes that the stick is being held steady in whatever position it was last in.
Well, at the time we lost the joystick signal, the right track was going full bore while the left was stopped (a hard turn). We knew about the joystick problem (we have multiple fail-safes in every other stage of the process, but can't do anything about joystick encoding), so the second I lost command control, I checked the cable. The problem was that it wasn't unplugged -- just loose. If it had been unplugged, I would have plugged it back in. That was the first problem.
The second problem was what really became evident. As the robot was sitting there doing doughnuts in the parking lot, one of the tensioner arm wheels came off. This caused the track to bite into the arm, and seize against the motor (which was at full power). This had the neat little effect of finding the weakest link in the drive train very quickly, and, just as I suspected, it was the plastic of the snowmobile sprocket. The steel slab that I had added this morning basically acted like an egg beater, and a big cloud of vaporized plastic came pouring out the side. This was the point where I made a run for the robot and hit the main power switch.
Ok, it was time to take stock of the situation. We had a destroyed snowmobile sprocket, which isn't that big of a deal because they weren't transmitting as much torque as I wanted anyway. This gives me a really good excuse to go buy some better ones. However, the computer is behaving strangely. We started the system back up to see if we could use the left track to help load this big, 400 pound paper-weight back into the car. I eased the power on, a little more smoke came out (this time out of the speed controller), and the whole thing shut down. Bad sign.
We couldn't get it working, and the parking lot wasn't the ideal place to work on it, so we loaded it in the car with brute force, intent on working it all out once we got home. There was clearly something wrong. The controller's output statistics were outright lies -- it said that it's voltage was 55 when I knew it was 12, it said that the current was 200 or more when there was really none, and it said that the temperature was negative. We could, however, communicate with it. We upgraded the firmware and were even able to save our own custom settings to it, but it still never behaved right.
Finally, we took the whole thing apart. Afraid that there was a wiring problem we couldn't see, we set about tearing apart the entire robot in order to completely isolate the speed controller. When this was finally done, we were able to open the speed controller and see inside. What we saw were scorch marks. The speed controller had fried.
For the record, this is practically a worst case scenario. The speed controller is the single most expensive component in the entire system. I paid so much for it because it has current protected circuits -- if it ever starts to overheat, it will throttle its own power to save itself (or, at least, it was supposed to). Just like the batteries that couldn't spill (which did), this thing could not cook (and it just had).
I tried to contact RoboteQ, but this ended up being somewhat difficult. It was only after two emails were sent, a tech-support form was submitted, two phone calls were made, and finally a plea for assistance was posted on their forum that they finally got back to us. They told us that the controller should protect itself against what had happened. My only theory was that the regenerative breaking during that fraction of a second before the motor spun down had no batteries to go to, but they told me that this shouldn't cause damage. They offered to give me $59 for the scrap if I sent them the thing. I was somewhat less-than enthused with this offer.
So, for the $500 I spent on the last one, I got, all told, something like 30 minutes of actual use out of the thing. Way not impressed. The only reason I paid so much for the thing was because it wasn't supposed to burn out. Apparently RoboteQ's products don't live up to their advertising. We've got less than two weeks left before we have to present this thing, so there isn't anywhere near enough time to modify the software for a new controller. A new RoboteQ AX2550 (exactly like the burned out one) has been purchased, and is in the mail as I type this. I'm not happy about it (normally whenever I'm treated like this, I never give the company another dime), but it's the only way I'll have any chance of getting the robot up and running in time for the presentation.
Update (04.02.2005): Last night, a representative from RoboteQ called me, and we talked for over half an hour about my problems. He had checked out this site, and had some thoughts on the way I'd set up the electrical system. He still wasn't able to tell me what had happened, and they haven't taken any responsibility for the failure, but at least I got some information. He had some good suggestions on how to protect the new controller against a similar fate. I'm going to change some of the acceleration ramping settings to safer values more suited for Etek motors, I'm rewiring the system with longer, thinner wire between the controller and the motor to give a slight increase in resistance and inductance, and I'm making a few other minor changes. I'm still not happy with the AX2550 -- this information didn't do anything to change the fact that the first controller didn't survive the most basic tests and I'm still out over $1000 on speed controllers. However, it is nice to know that when pressed to, RoboteQ can provide some support for their product.
However, it is worth mentioning that another company earned my spite recently: NPC Robotics. I sent them an email on Wednesday asking them for information on possible causes for the AX2550 (they're the vendors that I purchased both the original and the replacement from), and I haven't heard a peep out of them. On the same day, I placed my order for the new controller. I got the automatic response on the order, but here I am, three days later, and I have no shipment confirmation (I paid for second-day air) and still no response to my initial inquiry. No one seems willing to talk to me about this damn controller unless I spend a week pestering them first.
Update (04.12.2005): NPC did come through, sort of. I called them the day after the controller was supposed to arrive at my door, and they informed me that it was out of stock. This was not good news -- I was under a very hard deadline and it was soon. However, on the phone, they owned up to the problem (that goes a long way in my book), and worked with me to get it taken care of. They found out where their restock shipment was, when it was going to get there, and overnighted my controller to my door ASAP. It got to me 28 hours before I had to go give the final presentation, but that was enough in a pinch.
Of course, that wasn't the end of the problems. The new controller went into the bot, we got a few hours of sleep, and then headed off for our final presentation. We had some minor battery problems, but we got through the thing, the robot worked, it moved, everyone was impressed. On the way back to the car (read, the day after the controller got here, maybe 10 minutes of driving total on the thing) the new controller smoked. I said some choice words, removed the tracks (which is actually easier when you're blind with rage), and loaded it into the car.
So, emails went out, and back on to RoboteQ's forums we went. They made an effort to discredit me (I was starting to make them look rather bad), but finally did admit the problem ("The controller has current limiting, however this feature cannot be depended on to overcome such a mismatch," referring to the Etek motors). For returns, I've been talking a lot with NPC Robotics, who have been great. They've agreed to refund my money on the first one and are currently fighting with RoboteQ to get me the money for the second. It's finals week, so I'm perfectly happy to let someone else do my fighting for me for a bit.
This does, however, bring us to the main point of this update. I found today that RoboteQ had deleted our forum thread. Apparently the reason that their forum contains only glowing praise is that when there is a problem, they remove it. This isn't really surprising, based on the way they've dealt with me up until now, but I still find it incredibly disturbing that this company, when they find a problem with their product, tries to hide it from their customers just because it doesn't make them look good. In any case, I went diving through my cache and found the thing, and I wanted to make it available. Of course, none of those links are going to work, and many of the pictures are broken, but the text is the main thing.
|