ESP32 auto program fix

Out of the box my ESP32 dev board required me to hold down the BOOT button at the start of programming to get it to program from the Arduino IDE. The board has a pair of transistors to connect the DTR and RTS lines to EN and GPIO0. This is intended to enter programming mode automatically but seems to have timing problems. If I don't hold the button I get:

Fatal Error Occurred: “Failed to connect to ESP32: Timed out waiting for packet header”

Looking around the internet I found a recommendation to add a 10uF electrolytic capacitor between EN and ground. While this did work for programming it seemed to inhibit the board reset at the end of programming. I found 1uF worked better, which let me use a small ceramic SMT cap, which is a lot neater.

The nearest ground is the top right pin of the module and I could have angled it to reach that but I found it easier to just link it to the metal can.

Investigating why it doesn't work I found the setup and hold times for the strapping pins in the ESP32 datasheet:

To enter programming mode GPIO0 needs to be low as EN transitions to high and stay low for 1ms after that.

This is the two transistor circuit on my board which is labelled ESP32 DEV KIT V1, which implements the truth table shown.

The truth table shows that EN and IO0 can't go low at the same time, so the zero setup time cannot be achieved unless both lines change together. Here is what it looks like on a scope after I added the capacitor.

During reset RTS is low and DTR is high, which makes EN low and IO0 high. At the end of reset DTR goes low and then about 1ms later RTS goes high. This would work without the transistors but with them EN starts to go high 1ms before IO0 goes low, so it violates both the setup and hold time. Adding the capacitor delays EN long enough for IO0 to be low as the chip comes out of reset.

My best guess is the transistor circuit is there to ensure random serial monitors don't put the board into programming mode if they assert both DTR and RTS. To work without a capacitor RTS and DTR would need to change state simultaneously. There is a long discussion about it here: https://github.com/espressif/esptool/issues/136.

Third Hand

A problem I often face hand soldering surface mount components is keeping them in the correct position while soldering the first end. The surface tension of the solder invariably moves the component.  When using paste and an oven the surface tension is the same on both ends and pulls the component into position. Solder paste and hot air works similarly but it is easy to blow the component away.

One solution is to tin the pad and the end of the component first and then use tweezers to hold the component on the pad while applying the soldering iron to the tinned end. However that requires a steady hand and today, after having two strong cups of coffee, my hands are far from steady!

Some time ago I bought a PCBite probing kit from Elektor. This consists of a steel base, four magnetic clamping posts to hold a PCB and four sharp tip probes with magnetic bases and flexible swan necks intended for making measurements. It works well for its intended purpose although I think the flexible necks would be better if they were stiffer.

I found they make an excellent third hand for holding SMT components in place while soldering.

Here it is holding an 0805 resistor, they are the smallest parts I am comfortable soldering by hand.

Sinkholes

Continuing my series of adventures in 3D printing holes the correct size, I turned my attention to countersink holes. 

I don't normally use countersunk screws in 3D printed parts because they create lateral stress, whereas other forms of screw exert a purely compressive force, which can be spread by a washer. However, I am currently designing a part that needs flush screws in its base that is only 4mm thick, so there is no room for a counter-bore to bury a flat head.

First I had to improve my countersunk screw model because I previously just used a 45° cone with a sharp edge. In practice the edge has a nominal thickness of 1/10th of the screw diameter. That means if you countersink the hole with a normal conical countersink drill bit it has to be oversized to sink the head flush, and then leaves a gap around the circumference.

So the ideal shape for a 3D printed hole is an improvement because it can have a straight section at the top.

The complications are that, like horizontal holes, we have to take into account the rounded staircase produced by the filament layers and also, like vertical holes, we need to use polyholes to get the correct diameter when printed.

The cone makes contact with the rounded filament edge above, rather than the centre of the extrusion, where the slicer samples the model, as shown below:

In order for the extrusion to not overlap the cone it has to be offset outwards by a distance $x$, which is simply $(\sqrt2-1) r$ where $r$ is half the layer height. Applying that to the line equation of the cone and limiting the result to be in the range between the screw clearance radius and the screw head radius gives the correct placement for the rings of filament that just touch the cone without overlapping.

To make each of the rings print the correct size I make a stack of polyholes to subtract from the model.

These are double sided so they can be subtracted from the either the top or the bottom of a hole. As the slope is 45° they should be printable either way up.

This is what it looks like subtracted from a part:

I have added an OpenSCAD module called screw_polysink() to NopSCADlib to make printed countersinks. I haven't tried printing it yet but it should certainly work on a top surface. It might get a bit scrappy printed as an overhang without support.

Over cooked it

When I was printing 24/7 for about 5 years I never had a problem with filament absorbing moisture because the heat from the machines kept the rooms they were in hot and dry and the filament was stored in the same rooms as the printers. In fact in winter it was so dry I had to buy a long conductive ESD mat that runs the length of my workshop and wear ESD ankle straps to prevent getting sparks off everything I touched.

Since I retired I print more sporadically and tend to go away in winter and leave the heating at only 12°C. I still didn't have a problem with the white ABS that I got from Germany for kit production. It was unusual in that its natural colour was white instead of cream and it didn't smell much while printing. I used to find that I might get one or two bubbles in the skirt round the object but after that the rest of the print was fine. It appeared that filament sitting in the extruder from the last print would absorb some moisture over time but the rest on the spool that hadn't been melted did not, no matter how long it was left.

I also got some black ABS from the same German company and that is totally different. It bubbles very badly and needs to be dried. It also smells like ABS when it is printed. I never got good results from printing it, so I put off using it for years until I ran out of white. I then decided to tackle the moisture problem. Inspired by RichRap's heated dry box, I designed a parametric heated dry box that I could tailor to fit any size spool.

When I made the Dibond version of Mendel90 I noticed the dummy load resistors for the ATX PSU ran a lot cooler than they did on the MDF version. I came to realise that Dibond makes quite a good heat spreader even though the aluminium layers are only 0.3mm thick.

I also had lots of 47 Ω 50W TO220 resistors from various heated bed iterations that didn't go too well. Since this doesn't need to get very hot or need much power I thought it would be a good way to use them up.

I have a parametric box in NopSCADlib that is made from Dibond panels and printed brackets that can be scaled to any size that fits my CNC mill, so it was easy to wrap that around my large 2.4kg spools and add 12 resistors along three sides. The spool runs on 608 ball bearings between penny washers, there is a thermistor to monitor the temperature and a small fan to stir the air around.

I ran the resistors in series from half rectified mains to give a total wattage of 51W. I earthed all the panels and covered all the connections with heat shrink sleeving but it wouldn't pass any safety standards as the wiring isn't double insulated. Safe enough for me though as I don't need to put my hands inside it but I wouldn't recommend it. 

For my smaller 1kg spools I use 9 resistors and wire them in parallel for 12V operation. That gives 28W, which is enough for the reduced surface area and much safer.

To reduce the energy consumption I obviously needed to insulate the outside of the box, so I suspended it inside another slightly larger box and filled the gap with cotton wool. It is a lot nicer to work with than fiberglass or rockwool and quite cheap in the quantities needed. It is also just the right thickness to fill the gap, which is dictated by the corner fixing blocks. Even without the cotton wool the air gap gave quite good insulation. The only connection between the inner and outer cases is a few printed standoffs and the wires and filament exit guide.

It is completed by a hinged double door made of acrylic sheets with a hygrometer and thermometer module in the inner door. An arduino Leonardo with an LCD, some buttons and a couple of MOSFETs controls the temperature and the fan and keeps a record of the heater duty cycle. Again parts I had to use up.

So basically I created a 3D printed fan oven! 

The base of the outer box is extended, so it bridges the frame stays of my Mendel90s after removing the spool holders. Four fixing blocks stop it sliding off but are only screwed to the base, not the printer., so it can be lifted off and they then act as feet.

The mains version is controlled by a stand alone Arduino thermostat I had previously built to control a beer fridge. Another reason the first version was mains operated.

I have parts to make a third one the same as the second, after that I will probably make it headless with an ESP8266 and an I2C temperature and humidity sensor.

To dry the ABS filament I set the temperature to 80°C for a few hours and then left it at 50°C, even when the printer is not in use. The relative humidity in the box drops to about 19%. In the room it is about 60%. When I was printing 24/7 it used to be well below 50%.

As well as stopping bubbles it improves surface finish making it more glossy, makes bridges pull tighter, completely stops nozzle ooze at the end of a print and even reduces the ABS smell while printing to almost nothing.

At the end of a print I retract an extra 1mm and turn off the heater before moving Z back to the top. Without the dryer I used to get 10 to 20mm ooze out of the nozzle as the extruder cooled down. I had always assumed this was due to gravity but it is in fact due to moisture turning to steam pushing the filament out. When it is dry the surface tension must be sufficient to stop any flow due to gravity.

I was used to snapping off the ooze before I start a build, it had become an unconscious action, but now there is nothing to remove. It will be a big advantage when I make a multi material machine as there should be no ooze from the idle extruders while another is being used.

The reduction in smell was a complete surprise. Before I dried the black ABS it seemed to smoke as it came out of the nozzle. I now realise that was water vapour and it must carry off some volatile products. It now hardly smells at all when printing but if I stick my nose in the dryer it does smell of hot plastic, even though it is only at 50°C.  I guess the moisture it has driven off carries some VOCs with it.

About a year ago I got some wood coloured ABS in the UK which was even more affected by moisture than the black, so that is when I built the second smaller dry box, around October. After drying it printed very well. I made this replacement handle for a curtain puller with it.

After switching the dry boxes off when we went away for the winter I turned them on again in March and they have been on ever since. I last printed with the brown plastic at the end of May and it was fine but when I tried to use it yesterday it has gone super brittle. The coil has a very strong heat set, something I don't like about 3mm filament on 1kg spools, and bending it straight now causes it to snap.

I was trying to use it on my original MDF Mendel90 that is now encased in a box with a chamber heater. The filament feeds from the top of the box rather than through a PTFE tube that runs all the way to the extruder. The part that was outside of the dry box for about seven weeks was still ductile but had moisture in it. When I pulled the dry part from the box through it just snaped, so I can no longer print it on that machine. 

It does still seem to work on my Dibond machines that do have the PTFE tube all the way to the extruder. That keeps some of the coil's curve and doesn't require it to be fully straightened. 

And the printed objects don't seem to be brittle at all. I haven't done any proper strength tests but a quick test bending a small 100% filled block with pliers it bent and got white bruises like ABS normally does.

So keeping brown ABS at 50°C for a few months seems to completely denature it but a melt cycle seems to restore it. I have heard of PLA going brittle on the spool but nobody seems to know for sure what causes it. PLA is also somewhat brittle but ABS isn't at all. Presumably the long polymer chains must get shorter somehow and then reform when melted. I am not sure if the temperature is the problem or if it is too dry. I have read there is an optimum moisture level for processing plastic at, rather than as dry as possible, not sure why.

The black ABS hasn't gone brittle yet and it has been in its heated box for longer.

My prototype MDF Mendel90 runs with a chamber temperature of 45C and I have noticed that the printed parts it is made of seem to become brittle over time. The extruder runs a lot hotter of course and the Wade's block tends to crumble after a few years and needs to be replaced regularly. They last a lot longer on my unboxed machines. 

I also think ABS shrinks over time, even at room temperature. I made a test print with some holes in it a four years ago when I was having an interesting discussion about Polyholes with Giles Bathgate. I tested it with plug gauges but as I didn't have a case for them I left them standing in it on a shelf near a north facing window. When I went to use them for my Horiholes test I found they were stuck in it much tighter than I remembered and the plastic has yellowed slightly.

I made a case for them with a screw top using my thread utility in NopSCADlib, printed in the black ABS.

So now I have dropped the temperature of my dry boxes to 30°C as surely that won't degrade the plastic much more than in a hot room without sunlight. That vastly reduces the power consumption of course. 50°C needed about 50% duty cycle but 30°C is only 4%, only a shade over one Watt. The hygrometers are still reading 19% after a day at the lower temperature. I find that odd because, for a given air water content, reported relative humidity should increase as temperature falls.

So I think I will try initially drying new plastic at 80°C overnight and then reducing to 30°C for long term storage from now on and see how that goes. A good measure of whether it is dry enough is the complete lack of ooze at the end of the build.

Horiholes 2

As whosawhatsis pointed out in a comment on my last post, the edges of the filament staircase are actually semicircular and that makes a big difference as to where they should be to meet the circle tangentially. Not sure why I missed that as I have done several posts about extruded filament shape, I must be losing it!

This is what my previous shape actually produces.

The circle only touches at four points. The correct shape is obtained by calculating where the semicircles meet the circle.

The centres of the ends of the filament lie on a circle with a radius of the hole plus half the filament height. The end of the filament is then offset inwards horizontally by half the filament height. I.e. the slicer samples the layer at the central tip but the filament touches the circle on a tangent.

To make the shape geometrically I make a teardrop with a radius of the hole plus half the filament width, split it in half and shift the two halves together by half the filament width. As can be seen here that goes through all the filament tips, i.e., where the slicer samples.

This is what it looks like relative to the target hole.

Interestingly it is the same as my previous attempt at the top, bottom and sides, i.e. the only four points it touched before.

I made a test with the new formula.

The plug gauges all fit, but more snugly than before, so this is definitely a better solution for supporting a bearing in a pocket. At every layer it should have a tangential support from the rounded edge of an extrusion.

Here is a close up of the 6mm hole that is aligned on a layer boundary and the 1mm hole above it.

I have updated NopSCADlib on Github to use this method.

Horiholes

Back in 2011 I came up with polyholes to get around the problem of 3D printed vertical holes coming out too small. Horizontally printed holes also come out too small but for a different reason: the slicer creates a staircase approximation of the hole, but because it samples the model at the middle of the layer, the top or bottom corners intrude into the circle.

 This shows a 6mm truncated teardrop sliced with 0.25mm layers, with red highlighting the overlap.

A long time ago I mitigated this by adding 1/4 of the layer height to the radius, for reasons I can't remember now, but it isn't very accurate. There is some slight interference vertically and gaps at the sides.

I am currently designing a gearbox with ball bearings in 3D printed pockets, which I want positioned accurately, so I decided to revisit the problem.

The correct solution is simple: the compensated shape for the teardrop is simply the hull of itself shifted up half a layer and shifted down half a layer. That compensates the top half so that the bottom of the layer ends up on the circle and the bottom half so that the top of the layer ends up on the circle.

So now all the tips of the stairs sit exactly on the circle, except near the top where the 45 degree overhang would be exceeded.

This is what the hole looks like before it is sliced.

It does of course make the model specific to being sliced at a certain layer height, but my models tend to be designed that way anyway,

I printed this test piece with 6mm holes at different offsets from the layer boundary as well as holes from 1mm to 5mm and I tested it with plug gauges.

The gauges fit all the holes easily. Some are snug and some have a little play vertically depending on how the top edge aligns with the layer boundaries.

The bridge layers over the top come out a bit low because the filament forms a cylinder from a volume that would normally almost fill a rectangle, making it a slight interference fit when the top  lines up exactly with the layers. In other cases there is a bit of vertical play due to the 45 degree limit at the top of the teardrop. This won't be an issue with my gearbox because it will be split into top and bottom halves and the top half will be printed upside down.

I have updated teardrop_plus() in NopSCADlib to use this method and also added a plus option to all the other variants like tearslot(). See https://github.com/nophead/NopSCADlib#Teardrops and https://github.com/nophead/NopSCADlib#Horiholes where you can find the code to make the test STL. 

It should work universally as long as all slicers slice in the centre of the layer. Obviously it makes less difference with smaller layer heights.

NopSCADlib

When I used OpenSCAD to design Mendel90 I modelled complete assembly views with all the vitamins in place and a significant part of the code was actually the vitamins. Vitamins being the RepRap term for non-printed parts of a 3D printer, fasteners, motors, etc. I also automated the generation of the bill of materials, STL files, DXF files and PNG assembly views, but making the build manual was still a lot of manual work, pardon the pun.

After Mendel90 production ended I started designing other projects and found myself needing to use its vitamin library, but because it wasn't designed to be stand alone, that quickly got messy. Eventually I made a new stand alone library and a more general Python framework that would work for any project.

Over the last few years I have refactored it many times, making it much faster to preview, more general and more automated. In particular it can now catalogue all the vitamins and automatically make build manuals for any project using Markdown embedded in OpenSCAD comments. There is a simple example here. I also added reusable printed parts, such as feet, hinges and handles and some reusable enclosures.

It will never be complete because each significant project I make with it usually needs a few more types of vitamin, but it is hopefully structured so that it can grow sustainably without bringing OpenSCAD to its knees. To do that I had to fix some issues in OpenSCAD itself because it used to slow down exponentially with the number of files used. The picture above has an instance of every part in the library. The latest release of OpenSCAD can draw it in about one minute on my desktop PC. This is remarkable because at one time it took 12 minutes.

Here is an example of a typical assembly views it creates: -

I have published it open source on GitHub to enable me to publish projects that use it in the future. I use it for every project I make now, so I don't have any stand alone scad files that can be put on Thingiverse, for example. Feel free to use it in your own projects.

Beware fake multi-meter leads

I bought these multi-meter leads on eBay for £2.99. They are advertised as "16PCS/Set Multimeter Probe Pin Test Leads Cable Multifunction Digital Clip Kit". I was attracted to them by the large number of accessories including pin type plugs that fit old analogue multi-meters.

When they arrived I discovered that the wires are steel and have a resistance of about 1 Ohm each, which makes them about as useful as a chocolate teapot. Any resistance measurement gets 2 Ohms added . Current measurements on the amps range drop large voltages, the wires get hot and would burn if left on for more that a few seconds.

Voltage measurements would be accurate enough but where the attachments screw on there is exposed metal, so not suitable for high voltages.

I got a full refund and get a set of these instead for £2.97.

They claim to be CE cat III rated for 1000V and 20A. Their resistance is only 64 milliohms. Not bad but my UNI-T UT61E came with 600V 10A rated ones that measure 44 milliohms and my EEVblog BM235 wins with 24 milliohms for 1000V 10A probes.

I got them to replace these old ones that belong to a multi-meter I inherited from my Dad. They have numerous burns from touching a soldering iron.

I dug out this old meter when I realised all my modern digital meters only go up to 1000V at most. This one goes up to 6KV!

I think it was purchased sometime in the early 1970s. It is branded Honor Model TE-12, made in Japan. I have seen identical ones on the web branded Lafayette. I don't think you would get away with connectors like that for 6KV nowadays!

Avoiding annoying Blogger cookie warnings

Being warned by every web site that it uses cookies is very annoying. I don't see how it is useful because virtually every site uses cookies.

When looking at a Blog hosted by Blogger it is even more annoying because it warns for every new page of a blog visited. Looking at the cookie it uses to decide whether to warn about using cookies I noticed it stores the path of the page relative to the blog. That is why it warns for each page of someones blog. However it looks like it won't warn for sub paths of the path in the cookie. So if you visit the root of the blog and dismiss the warning there it gets rid of the warnings for every other page in the same blog. This is a lot more reasonable and probably the way it should work by default.

It is also annoying that this will expire in a year. If every website I visit repeats the warning every year it will continue to be a constant stream of warnings forever.

ESP8266 SPI Spy

I came across a very useful post by Thomas Scherrer that describes how to read data from a Peacefair PZEM-021 energy meter by spying on the SPI bus with an Arduino. I decided to do the same thing with an ESP-12F WiFi module so that I could view the results remotely and plot graphs, etc. It took me a lot longer to get this working than I anticipated due to a few problems along the way.

The main hardware difference is the ESP8266 is a 3.3V device but the Arduino is 5V. The PZEM-021 is actually a mixture. The RN8208G metering chip is a 5V device. It is a SPI slave, the SPI master is an STM32 ARM processor that is 3.3V but with 5V tolerant inputs. That means the signals originating from the CPU can go straight into the ESP8266 but the data out from the RN8208G would need attenuating.

Copying Thomas, I removed the mains dropper components C1 and R1 and powered the PZEM-021 from an external supply. This allows it to measure voltages right down to zero instead of cutting out at 80V. I used this little 12V 1A supply recommended by Big Clive on YouTube.

I powered the ESP-12F from the same supply with a tiny 3.3V MP2307 buck converter module.

WARNING: when connected to PZEM-021 the 0V rail is at mains neutral potential. This has to be treated with the same precautions as live because if the mains lead happened to be be reversed or the neutral connection broke it would become live. Don't connect a USB programmer or a scope ground to the circuit unless you power it with a mains isolation transformer. Fortunately the ESP8266 can be programmed wirelessly from the Arduino IDE once a sketch with ArduinoOTA has been installed and this is much faster than USB.

During the hardware development I used a mains isolation unit that I made after watching this video by Paul Carlson, another of my favourite YouTubers. I built this originally to allow me to repair a switch mode PSU that was part of a friend's home cinema unit. It allows me to connect my scope to the live part of a circuit but then of course it is no longer isolated, so great care has to be taken not to touch the high voltage bits.

It is a DiBond box with 3D printed frame, handles and rubber feet containing an isolation transformer and a small variac. It also has two 100W light bulbs in parallel with a bypass switch for optional current limiting, a different model of power meter and some 4mm jack sockets allowing me to attach my Mooshimeter to log voltage and current.

The first task was to solder wires on to the PZEM-021 PCB to bring power in and data out to my circuit.

I didn't see a need for the chip select / word latch signal and I took the data from the other end of R9 compared to Thomas. The signals are as follows:

• Red is 12V, black is ground. 
• The word latch going to the display from chip U2 pin 7 on the left, used for synchronisation. 
• MISO data coming from the energy chip via R9.
• The SPI clock coming from U2 pin 12.

The two wires on the chip are tricky to solder due to the fine pitch, which is much smaller than a practical soldering iron bit. I used the following technique:

• Strip some wire-wrap wire and trim the end to be the length of the flat part of the pin.
• Tin it. It doesn't hold much solder due to being such a small radius.
• Add plenty of liquid flux around the pin.
• Line the wire up along the top of the pin with an Andonstar ADSM201 microscope camera.
• Put a very small amount of solder on a small chisel bit.
• Press the iron on top of the wire and leave it long enough to conduct the heat through the wire to boil the flux and melt the solder on the pad below.

The amount of solder on the iron bit is crucial. If you use too much it will bridge the pins. To remove a bridge add plenty of flux and wipe the pin with a large flat bit. Surface tension will cause some solder to wick onto the bit, wipe it off and repeat until the bridge breaks.

When I looked at the levels of the signals I got a bit of a surprise. I was expecting MISO on channel 2 to go to 5V but the other two come from the ARM and should be 3.3V signals.

They seem to have a 20kHz ripple that takes them up to 4V. I looked back at Thomas' oscilloscope pictures and see the same ripple there. It doesn't make any difference for him because he is using a 5V chip but I didn't want to stuff 4V into my ESP8266!

On further investigation I found that the whole 3.3V rail had this ripple on it. It comes from a Holtek HT7133-1 3.3V LDO regulator. The datasheet for that suggests 10uF decoupling capacitors on the input and output. The circuit has what looks like a 10uF tant on its input but the output decoupler is a tiny MLCC that looks too small to be 10uF. I added a 10uF electrolytic in parallel and that killed the oscillation. It is the blue radial cap across C1 in the photo above.

That just left the issue of the 5V signal to contend with. R9 turns out to be 1K so simply adding a 2K2 to ground drops the signal low enough.

It also changes the first eight bits from FF to 00. This is actually when the command byte is being sent to the meter chip on MOSI and MISO is tri-state, so it doesn't matter.

Hardware done, on to the firmware. Before starting the project I had seen that the ESP8266 has a spare SPI port available and I assumed it would be straightforward to just read a stream of 13 bytes, wrong! It wasn't because the SPI port it a relatively complex device with dozens of registers that doesn't just send and receive bytes. It actually works at the command level, expecting to send or receive a command and address and then send or receive status or data bytes.

When in host mode it is possible to configure it to send and receive arbitrary bytes, indeed that is what the Arduino SPI class does and it works on the ESP8266 more or less the same as it does on an AVR. In slave mode though it has to receive a command and address according to the technical manual and the command defines what happens next. The relevant sections are:

4.3.2. Communication Format Supported by Slave SPISlave ESP8266SPI communication format is almost the same as that of the master mode, i.e. command+address+read/write data, but the slave read/write operation has itshardware command and undeletable address, which is,

• Command: a must; length: 3 ~ 16 bits; master output and slave input (MOSI).

• Address: a must; length: 1 ~ 32 bits; master output and slave input (MOSI).

• Read/write data: optional; length: 0 ~ 512 bits (64 Bytes); master output and slave input (MOSI) or master input and slave output (MISO).

4.3.3.Command Definition Supported by Slave SPIThe length of slave receiving command should at least be 3 bits. For low 3 bits, there are hardware reading and writing operation, which is,

• 010 (slave receiving) : Write the data sent by master into the register of slave data caching via MOSI, i.e. SPI_FLASH_C0 to SPI_FLASH_C15.

• 011 (slave sending):Send the data in the register of slave data caching (from SPI_FLASH_C0 to SPI_FLASH_C15) to master via MOSI.

• 110 (slave receiving and sending): Send slave data caching to MISO and write the master data in MOSI into data caching SPI_FLASH_C0 to SPI_FLASH_C15.

On the face of it this looks like it will not work in this application, which is unbelievable that a SPI port can't read arbitrary data as a slave and is hard coded for emulating flash chips and the like. 

I couldn't find any proper register level documentation for the ESP8266 other than various .h files on the web. The Arduino library uses shortened names which are very cryptic but some headers use longer names. While searching for these I came across the technical manual for the ESP32 and realised it has SPI ports that are nearly the same. It also has register level documentation in it's technical manual so I was able to get a slightly better understanding. After a lot of experimentation I found a solution.

The first n bits get interpreted as a command, where n is 3 or more and defaults to 8. The start of the 13 byte sequence that I want to receive is always zero now that I added the pull down resistor. So it always receives command zero. The bottom three bits normally control what happens next and zero defaults to read status. However, it is possible to override this with user definable commands by setting  SPI_SLV_CMD_DEFINE in SPI_SLAVE_REG. That makes the SPI_SLAVE3_REG define what the command values are for four different actions. By setting SPI_SLV_WRBUF_CMD_VALUE to zero and the other three values to nonzero I was able to make it interpret the command as write buffer, so the rest of the data gets written into the buffer.

Well that was the theory but I was missing two bytes at the beginning. This was because it was being interpreted as an address. The default address length is 24 bits. I changed it to 8 by setting SPI_USER1_REG and now I get the 12 bytes of data following the zero byte. Now this doesn't make sense because if the first 8 bits are being interpreted as an address where is the command coming from? The command length is set by  SPI_USER2_REG and it defaults to 8. I was expecting to have to set it to say four and set the address length to four bits so that they could share the first byte but any value between 2 and 8 seems to work and makes no difference. I don't know if it is a bug in the chip or I don't understand something but I get to read my 12 bytes of data

So in conclusion you can't read a completely arbitrary SPI data stream but you can if the first few bits are a known value.

Here is my code that sets up the HSPI:

When the data is received I get an interrupt where I copy it from the HSPI's dword buffer into a byte buffer in RAM.

I set up the sync pin to generate an interrupt on the falling edge. This resets the HSPI and unmarshals  the data:

I found that when I switched the load on and off I sometimes got a data packet shifted right one bit. That points to a spike on the SPI clock line into the ESP8266 but I could not find one with a 100MHz scope triggered on the mains switching edge. I added a snubber close to the switch but that didn't cure it, so I bodged it in the firmware and moved on. If a voltage sample is not more than half the previous one I ignore it once.

Another oddity is that when there is no load and the voltage is relatively high the power reading would go slightly negative. I detect that and set it to zero.

The foreground loop looks for the new_readings flag to be set and does the conversion to real units:

The conversion constants are close to what Thomas used but slightly different, so the ARM must have some per unit factory calibration constants in it.

So it took me longer than I anticipated to follow in Thomas' footsteps because of the complexity and lack of proper documentation for the ESP8266 HSPI peripheral but I got there in the end. I hope others find this as useful as I found Thomas' work.

By now you are probably wondering what this is all for. Well, while investigating a strange line regulation problem in a power supply I got annoyed that the mains voltage in our house is constantly varying, making it hard to make measurements of input versus output. This is what it looks like logged every 10 seconds over a couple of days using my Mooshimeter.

As well as the small modern variac shown above I have a old WM5 model that I got on eBay. The only data I could find on it was from 1955. It still works fine so I decided to automate it with a small stepper motor to maintain a specified voltage. I.e. make a WiFi control IOT variac.

Another DiBond and printed part creation but the front panel had to be acrylic to let the WiFi out. Fortunately the front of my bench faces towards my router. The ESP8266 controls the motor via a Pololu stepper driver.

The control interface is a simple web form that also shows the current readings using AJAX as described here.

The control algorithm is very simple. The motor is stepped at the specified speed by the error in voltage multiplied by the gain. The readings update every 200ms but they lag a lot. That causes overshoot if the gain is set high. It would probably benefit from a full PID controller but it works well enough for now. The deadband setting allows it to stay in a range without constantly moving because I don't want to wear out the variac. Servo control of a variac is a standard way of regulating mains but I have no idea how long the brushes last. Manual mode disables the motor.

The readings can be read and the settings changed from the command line or from a Python script using cURL. Here is an example that sweeps the voltage from 10 to 250 in 10 Volt steps and reads the current and power.

Here is a graph showing current, power and resistance versus voltage of a 60W light bulb

I plan to use this to plot a power supply line regulation graph. For that I will need a way of reading the output voltage in my script. I have two scopes that have network APIs as well as my Mooshimeter that has a BLE interface. I just need to work out which is the easiest to access in Python.

Here is the full ESP8266 sketch.