April 18, 2020

Titanium anodizing experiment

A month or two back I tried out anodizing a titanium bicycle fork.  Eventually I want to anodize, mask, and bead blast a frame along the same lines as Firefly , so the fork was a test-run to get a feel for the anodizing and masking process.

I did the anodizing by soaking a foam brush in a water and baking soda mixture, clipping one power supply lead to the brush, and one to the part, and brushing the part. 

If you're good at this, you can get nice gradients by adjusting the power supply voltage as you move along the part, but I definitely need more practice.  I love the purple and dark blue parts: 

I used some vinyl tape to mask out a pattern, and then bead blasted to strip the anodizing where it wasn't masked.  I didn't put any effort into this pattern since this was a test, but eventually I'll make a mask on a CNC vinyl cutter.  I need to find a blasting cabinet with a gentler media in it as well.

January 20, 2020

New High Power Motor Drive

I've been working on a large robot-size motor drive, with a design target of 75V, >70A (peak phase) continuous with heatsinking, and >150A peak.  In order to make the drive as compact as possible, it's split into three separate boards stacked on top of each other.  The bottom layer is the power layer with MOSFETs, ceramic DC link capacitors, and current shunts.  To minimize the thermal resistance between all the heat generating components to the world, the power layer is built on a 2-layer aluminum substrate board.  Above the power board is the logic board with microcontroller and gate-drive, built on a normal 4-layer 1 oz board.   On the very top is a breakout board with connectors for DC power input, the 3 phases, and communication/encoder

To get a better feel for how well the aluminum power board will work, I built a simple half-bridge using the same FETs as I plan on using for the final motor drive.  

Here's the first test-board, affectionately named The Half Bridge of Science.  It's just 2 parallel TPW3R70APL FETs per side, some 250V .47 uF ceramic caps, and a couple TLP152 gate-drive optocouplers to drive the gates.

Here's my dummy load for the half-bridge:  4 big "Pickle" resistors in a 2S2P arrangement for 1 ohm and lots of watts, with the boost converter inductor from a Ford Fusion hybrid in series.

Here it is with 55 DC amps (equivalent to 77 peak sinusoidal amps) going out of it at 50V on the input at a 50% duty cycle, basically perfectly maxing out the MITERS Lambda 50V 30A supplies.

Here are side-by-side shots of the board with a thermal camera at 55A switching at 10 kHz on the left, and 20 kHz on the right.  Here's it's poorly  clamped to a heatsink with no thermal paste.  At 10 kHz, the FETs (Sp2) ran at 57 C, and the ceramic capacitors (Sp1) at 55.  At 20 kHz, the FETs were at 65 C and the hottest of the ceramics at 75 C.  Looks like I need more DC link ceramics to handle the ripple current.

 There were a few mistakes with this board, so I expect thermal performance to only get better from here - I accidentally used the 1 W/m-K insulator instead of the 2 W/m-K insulator that's also available, and I misinterpreted the minimum via size, so the thermal vias were huge and sparse.  This seemed pretty promising, so I designed 3-phase version

Here's the populated 3-phase power board.  For testing, I got a normal FR4 board to confirm that everything works.  The copper power links between the boards were custom-machined, since I couldn't find any COTS pins that were as compact.

Here's the logic board, minus gate driver.  To start out, I'm trying the DRV8353S gate drive IC.  It's very similar to the DRV8323 I used on my smaller motor drives.  I went for the version without a built-in buck converter and use an external 15V and 5V buck to power everything.  Since the DRV8323 was good for driving 2 FETs very similar to these at 40 kHz, it should be able to drive twice as many FETs at 20 kHz.

The top of the stack has Wurth terminals to get power in and out, and some JST-SH's for logic.  I plan on replacing the logic connectors with something more robust like the Harwin Datamate L-Tek's I used on the Mini Cheetah, once everything is up and running.

The assembled stack:

Next time it will be a little bit thinner - I ordered the wrong male header for the board-to-board connectors, so the boards sit a little further apart than I originally intended.

Next to a coke can for a sense of scale.  The footprint of the drive is 45x45mm.

Hooked into a giant T-motor U15 I'm using for testing:

That's all for now.  There were two swapped pins on the logic board I had to fix with air-wires, but other than that everything seems to work.  Unfortunately I made some dumb testing mistakes and blew up the controller before I could push it too hard, and the stacked construction is almost impossible to disassemble.  But now that I know the board layout works I'm going to send out for the aluminum power boards.  Stay tuned.

December 15, 2019

Furuta Pendulum(s): Building some more

I built 6 furuta pendulums for a university controls class:

Since the original furuta pendulum was all built out of scrap I had lying around, I had to redesign pretty much everything from scratch.

Instead of using a hand-skewed motor like the original, I used a custom-wind of a T-motor gimbal motor.  These have low enough cogging torque to work pretty well:

Each motor gets a Microfit connector crimped on the leads, and a hollow shaft retaining-compounded in, for the encoder wheel to attach to:

The spindle has a pair of bearings very lightly preloaded by a wavy washer, and a diametrically magnetized magnet in the back which is read by a hall encoder:

In the base is a US digital 20k count/rev optical encoder which the wires for the pendulum hall encoder pass through:

SPI to the pendulum encoder gets passed over a 12-pin slip ring, with each pin doubled-up for (hopefully) better reliability:

In the base there's a board with power switch, mode switch, DC input jack, USB connector for serial communication, and programming header on it.  The mode switch allows the pendulum control to be done by the on-board motor controller, or by taking commands and passing back sensor outputs over serial to a computer running the controller.

I didn't want any screws to show, so the top is held on by 4 magnets:

Next to the original:

November 29, 2019

Building All The Robots

Here's a photo-dump of some pictures from assembling the herd of Mini Cheetahs.

150 Stators:

150 Front actuator housings with output bearing installed:

50 upper legs.  The upper link was consolidated from 3 parts to 2 for the new robots:

For assembly I got these fancy Wera adjustable torque screwdrivers, so I (and people helping assemble) could repeatably torque all the screws:

A handful of the 450 planet gears with needle bearings installed:

150 rings gears with OD's post-machined to size.  Ready to be pressed into the housings:

A sea of sun gears:

Actuator with stator and gearbox installed:

I made several minor design mistakes I had to fix.  Some holes in the sheet metal were 1mm off, so I made this fixture for re-machining the holes.  Small mistakes like this sunk a huge amount of time, because each on has to be fixed between 50 and 150 times.  Fortunately everything was salvageable.

Most of the actuators with stators and planetary gearboxes installed:

Part of one robot body with computer, CAN interface, IMU, RC receiver, and power supply:

The glorious actuator cabinet:

Computers and CAN breakouts:

Power supplies, body center-plates:

Belts installed, ready for tensioners and outer half of the leg clamshell.

A mostly assembled leg:

A nearly complete robot, waiting for the last few wire-management steps and final body assembly:

One robot during electrical and software testing, before final assembly:

Half a robot's worth of actuators:

Final assembly step of attaching the legs to the body:

Robot with top panel removed:

A pair of robots ready for final testing:

Most of the new robots lined up on the lab treadmill:

October 31, 2019

Motor Temperature Estimation Without a Sensor

I just finished building a bunch of Mini Cheetahs which the lab  is loaning out to other research groups.  Since we're giving these to people who for the most part aren't hardware oriented, these robots need to be even more robust than the original one.

One piece of that is preventing the motors from burning out.  During normal operation with a good locomotion controller, the motors barely even get above ambient (even going 3.7 m/s, the motors weren't warm to the touch afterwards).  However, it's really easy to write a controller that rapidly heats up the motors without actually doing anything - railing torque commands back-and-forth due to instability, joints mechanically stuck against something, machine-learning algorithm flailing around wildly, etc.  We haven't burned out any motors on the original Mini Cheetah yet, but I think our lab members all have a pretty good sense of what's good/bad for the robot, and know to shut it down quickly if something bad is happening.  But when the robot is in the hands of a bunch of software and machine learning engineers.....  So to protect the robots, I'm adding in winding temperature sensing and over-temperature protection, to (hopefully) make it impossible to burn out the motors.

Now, the smart way to do this would have probably been to just add a thermistor in the windings, and call it done.  Obviously, I didn't do that, so here's my observer-based workaround.


The temperature estimate uses an observer to combine two sources of information:  A thermal model of the motor, and a temperature "measurement" based on the resistance temperature coefficient of the copper in the windings. The resistance of the motor is estimated based on applied voltage, measured current, measured speed and known motor parameters, and compared against the resistance at a known temperature.  Sounds pretty simple, right?

Of course not.  If it was, it wouldn't be worth a blog post.  It's not terribly complicated either, but it took a bunch of little hacks to make it actually work.

Thermal Modeling:

I'm using a 1st order thermal model with just thermal mass  \(C_{th}\) and thermal resistance to ambient  \(R_{th}\).  With temperature rise over ambient \(\Delta T\), thermal power going in \(\dot{Q_{in}}\), and thermal power going out \(\dot{Q_{out}}R_{th} =  \Delta T\), the dynamics are
\Delta \dot{T} = \frac{\dot{Q}_{in} - \dot{Q}_{out}}{C_{th}}
To get \(\dot{Q}_{in}\), I just use \(i^{2} R\), but accounting for the variation from nominal resistance\(R_{0}\) due to \(\Delta T\), i.e. it's temperature coefficient of resistance \(\alpha\) (.00393 for copper):
\dot{Q}_{in} = (1 + \alpha\Delta T)R_{0}i^{2}
In (slightly simplified) code, the model part of the observer is updated as follows, by Euler-integrating the thermal model.

delta_t = observer.temperature - T_AMBIENT;
observer.qd_in = R_NOMINAL**(1.0f + .00393f*delta_t)*controller.i*controller.i;
observer.qd_out = delta_t*R_TH;
observer.temperature += DT*(observer.qd_in-observer.qd_out)/C_TH;

An important implementation detail, in the actual firmware I'm doing the last line of math as doubles, rather than floats like everything else.  My sample period DT is very small (since my loops run at 20-40 kHz depending on the motor I'm driving), and  \( \frac{\dot{Q_{in}} - \dot{Q_{out}}}{C_{th}}\) is also pretty small since the thermal dynamics are very slow.  As a result, the change in temperature over one loop-cycle gets rounded down to zero when you use floats.  Since STM32F4's are crap-tastic at double math, this single line takes a substantial fraction of my 25 microsecond loop period when I'm running at 40 kHz.  I'm sure there's a way to do this avoiding doubles, but I have just enough computational headroom that I don't care.

Measuring Temperature and Resistance

Assuming we can measure the resistance of the motor perfectly and we know nominal resistance \(R_{0}\)  at some temperature \(T_{0}\), and the temperature coefficient  \(\alpha\) , we can calculate temperature:

T = T_{0} + \left(\frac{R}{R_{0}} -1 \right)\frac{1}{\alpha}

To measure \(R\), use one of the synchronous motor voltage equations.  I use the Q-axis one, because for my surface PM robot motors there's usually not much current on the D axis.

V_{q} = Ri_{q} + L_{q}\frac{di_{q}}{dt} + \omega L_{d}i_{d} + \omega\lambda

We'll assume \( \frac{di_{q}}{dt} \) is zero, since the temperature observer is going to be very low-bandwidth compared to the current control dynamics.  Conveniently if you use the Q-axis voltage equation rather than the D-axis one, the \( \omega L_{d}i_{d} \) term is usually zero, since \(i_{d} \) is only non-zero at high-speeds during field weakening, and in that region you can't get enough q-current into the motor to burn it out anyways.

Since we know \(V_{q}\), \(i_{q}\), \(i_{d}\), \(\omega\), \(l_{d}\), and \(\lambda\), we can just solve the voltage equation for \(R\).  In reality I had to add a trick to get this to work, but I'll get into that later.

Implementing the Observer

The basic steps in the observer are:
  • Integrate forward the dynamics of the quantity you are estimating
  • Take a sensor reading and calculate the error between the estimate and the sensor reading
  • Use a proportional controller to drive the estimate towards the sensor reading
In code the whole observer looks like this:

// Integrate the thermal model //
delta_t = observer.temperature - T_AMBIENT;
observer.qd_in = R_NOMINAL**(1.0f + .00393f*delta_t)*controller.i*controller.i;
observer.qd_out = delta_t*R_TH;
observer.temperature += DT*(observer.qd_in-observer.qd_out)/C_TH;
// Estimate Resistance //
observer.resistance = (controller.v_q - controller.omega*(L_D*controller.i_d + WB))/(controller.i_q);

// Estimate Temperature from temp-co //
observer.t_measured = ((T_AMBIENT + ((observer.resistance/R_NOMINAL) - 1.0f)*254.5f));

// Update Observer with measured temperature //
e = (float)observer.temperature - observer.temp_measured;
observer.temperature -= .0001f*e;

Naively implemented, the above didn't really work - the resistance measurements are terrible , so either you have a very low observer gain and basically run open-loop with just the thermal model, or the temperature estimate varies wildly depending on the speed and torque the motor is operating at.  It took a couple more additions to get it to work reliably.

Voltage Linearization

The first problem I notice was that the measured resistance changed dramatically as the current varied.  At low currents, the estimated resistance was much higher.  This problem was caused by nonlinearity in the voltage output of the motor drive due to deadtime.  I tried a few methods for compensating the dead time, but I got the best result by scaling my d and q axis voltages based on modulation depth.  I measured the nonlinearity by logging measured current vs commanded modulation depth over a range of currents, putting all the current on the d-axis so the rotor didn't move.

I generated a lookup table to scale the commanded voltages so that current vs commanded voltage is linear.  Around zero commanded voltage, I actually only get about .5 output volts per commanded volt, pre-linearization:

State-dependent Gain Scaling
i.e. lazy person's Kalman filter substitute.

Even with the voltage linearization, there are some operating conditions which make it hard to measure the resistance accurately.

  • At very low currents the measurements are bad.  You have a small voltage divided by a small current, so the result is super sensitive to noise or bias in either.
  • At high speeds the measurements are also bad since the flux linkage term in the voltage equation dominates.   Slight error in the flux linkage constant causes errors, and also non-sinusoidal flux linkage and inductances mean there's lots of ripple in the d and q voltages and currents at high speed.
The least sketchy way to incorporate these effects might be to figure out how the covariance of the temperature "measurement" depends on the operating state, and use a Kalman filter to optimize the observer gain based on the changing covariance.

My sketchy alternative was to hand-tune a "trust" function which behaves similarly to inverse covariance, to make the gain really small in states where I don't trust the temperature measurement.  Basically, if the current is below some threshold, make the gain really small, and/or if the speed is above some threshold, make the gain really small.  Around each threshold I have a linear ramp, to smooth things out a bit.  In code, my "trust" function works like this:

// Calculate "trust" based on state //
observer.trust = (1.0f - .004f*fminf(abs(controller.dtheta_elec), 250.0f)) * (.01f*(fminf(controller.current^2, 100.0f)));

// Scale observer gain by "trust" //
// .0001 is the default observer gain //
observer.temperature -= observer.trust*.0001f*e;

When the resistance estimates are the best (current greater than 10 amps, speed near zero), "trust" is equal to 1, so the observer gain doesn't change, and the observer gain goes to zero (i.e. open-loop) at really bad measurement points.  I'm sure this would horrify every controls professor I've had, but it works pretty well:

Here's a video of testing with one of the mini cheetah motors, with a thermocouple glued into the windings.  I'm railing the current command between -40A and 40A at 5 hz, so the motor only spends a very brief period of time at low speed, and most of its time at high speed with low current.  In this test I initialize the motor temperature at 25 degrees, even though the motor is still at 60 degrees from a previous experiment.  The observer takes about 16 seconds to converge to the actual temperature, but from there on it tracks to within ~5 degrees.  Once the temperature hits 120 degrees, the current limit is throttled back to 14 amps in order to keep the temperature below 120 C: