OpenWrt Forum Archive

Topic: Question: Debrick TL-MR3040 by flashing EEPROM chip directly

The content of this topic has been archived on 7 May 2018. There are no obvious gaps in this topic, but there may still be some posts missing at the end.

Hello. I hope I have posted this in the correct section of the forum.

Has anyone managed to recover a bricked TL-MR3040 (or similar router) by flashing the EEPROM chip using an SPI programmer? Or have any experience or knowledge of preparing & flashing a blank EEPROM when upgrading the flash memory capacity?

I have a bricked TP-Link TL-MR3040 v2.5. I purchased it earlier this year and successfully flashed it with OpenWRT via the web GUI to use with DSLR Dashboard. It worked perfectly for a couple of weeks, then somehow turned itself into a novelty paperweight. When I turn it on there is a brief flash from the LEDs, then just a solid power LED. I have tried the many times to use the failsafe mode/serial console recovery method, but without success - no meaningful text appears, just a blank screen or a few random characters. (I suspect something on the EEPROM has become corrupted, or possibly the chip has failed.)

So I would like to try and write a fresh firmware image onto the EEPROM (in this case a Winbond W25Q32FV) using FlashRom & a Raspberry Pi (3) as an SPI programmer. After some research I have an SOIC clip connected to the correct pins on the GPIO header of the Pi and FlashRom installed & executing successfully. Thus far all attempts to read the current contents of the EEPROM have resulted in a 4MB file of 'FF's - so either something is wrong, or I have found out why this TL-MR3040 won't boot.

The next step will be to try and prepare an image to flash onto the EEPROM. I have not attempted anything like this before, but I am aware that the EEPROM will contain not just the operating code, but also device specific information such as the MAC address & serial number. And that the flash memory is usually partitioned into different region. I hoped to have a dump of the current EEPROM contents to use for reference, but it looks as though that information has already vaporised. [I have a second TL-MR3040 (running OpenWRT), but I would prefer not to risk trying to make a dump of its EEPROM just in case anything goes wrong. This extremely versatile & useful device alas now appears to be completely unobtainable in Europe.]

So I am in need of advice & guidance on how to prepare an entire EEPROM image (based on either OpenWRT or the or the original TP-Link firmware) which can be flashed onto a blank chip (possibly a replacement, if the original is faulty).

Is there anyone who has upgraded their EEPROM or attempted a similar recovery who can help?

Thank you.


[I apologise in advance if the information I seek in readily available - I too hate it when people are too lazy to search for themselves and know the torment of having to answer the same old questions again & again which have already been answered dozens of times before on a forum. I have spent several months researching this as much as I can, but I have problems with my vision & a disability which makes reading large amounts of text extremely difficult.]

You'll probably have to desolder the flash chip for flashing, also at least try to recover the ART partition of *your* device (full backup strongly recommended).

(Last edited by slh on 31 Aug 2017, 03:54)

You can easily dump the mtd-partitions of your second device within OpenWrt using dd. No special risk involved.

Yes you should remove the chip from the router to try to read or write it.  With the chip installed, the router SoC and the Pi are both trying to drive the chip.

The software in the flash of a TP-Link NOR flash router consists of 3 major partitions.  The first 128kbyte is the bootloader.  The last 64kbyte is the "ART" partition with device specific information.  The center part of the flash is the firmware.   

The firmware space is internally divided into (for OpenWrt) a kernel image and two filesystems (one read only and one read-write), but you don't need to worry much about that.  An OpenWrt "sysuprade" image consists of the kernel and the read-only /root filesystem parts.  The read-write filesystem is created and formatted as OpenWrt boots for the first time. 

A very nice thing about the TP-Link OpenWrt builds versus most others is that they will detect the flash size and adjust to it.  You can simply drop a release build for a 4 MB model into a larger chip and have immediate access to the extra space.

Flashing OpenWrt is supposed to only change the firmware area.  OpenWrt contains some software write-protections for the other partitions. The bootloader of two routers of the same model should be the same.  The ART will be approximately the same but with differences in the MAC addresses and possibly the radio calibration.  Without a valid ART, the router can still boot and use bogus MAC addresses on the Ethernet, but the WiFi absolutely will not work.

To prepare a new blank flash chip for installation, first load OpenWrt and use it to download copies of your bootloader and ART.

cat /proc/mtd
dev:    size   erasesize  name
mtd0: 00020000 00010000 "u-boot"
mtd1: 00133024 00010000 "kernel"
mtd2: 00e9cfdc 00010000 "rootfs"
mtd3: 00c70000 00010000 "rootfs_data"
mtd4: 00010000 00010000 "art"
mtd5: 00fd0000 00010000 "firmware"

Almost all TP-Link models should have this same layout but it doesn't hurt to double-check.  Here we see that the u-boot is mtd0 and the ART is mtd4.

cat /dev/mtd0 > /tmp/uboot.bin
cat /dev/mtd4 > /tmp/art.bin

These two files are now on the router's RAM disk.  scp them to your PC for the next step.  The sizes should be 131072 and 65536 bytes respectively.  Check that they aren't all FF or 0.
Now you can assemble an image for the new flash chip.  Download the "sysupgrade" OpenWrt for your model.  You need to end up with (in a 16 MB chip, such as a W25Q128)
Offset 0x000000 - > uboot.bin
Offset 0x020000 - > sysupgrade.bin
Offset 0xFF0000 - > art.bin.  It is always in the last 64k of the chip regardless of chip size, so 3F0000 for a 4M chip and 7F0000 for an 8M.

The space between the end of the firmware and the ART can be filled with 0, FF, or anything.  It will be overwritten with the JFFS2 filesystem the first time the router boots.  With a 16MB chip, this first boot will take about 2 minutes to become ready due to the need to write flash.
After flashing this chip, solder it into your router and with any luck it should boot and run.  If you aren't real confident about that, you can solder some thin wires about 1 cm tall to the flash chip pads on the circuit board, then solder the chip to the top of the wires.  This makes it relatively easier to swap chips in and out.

(Last edited by mk24 on 1 Sep 2017, 00:25)

slh wrote:

You'll probably have to desolder the flash chip for flashing, also at least try to recover the ART partition of *your* device (full backup strongly recommended).

MBS wrote:

You can easily dump the mtd-partitions of your second device within OpenWrt using dd. No special risk involved.

Thank you both. That has given me a couple of very helpful search terms, and a lot more reading to try to do. But I think I'm beginning to make some significant progress with understanding the subject.

wiki.openwrt.org/doc/howto/restore_art_partition
wiki.openwrt.org/doc/techref/flash.layout
...& more.

Is it possible to dump all 5 into a single file (for study purposes) from the working TL-MR3040 using OpenWRT?

Also, is it known what is stored in the config partition? (Presumably the serial number & OpenWRT configuration settings are here.) But is there anything else of importance & unique to the device?

I see that the MAC address is stored in the ART partition. Is there a second MAC address for the Ethernet interface? (Stored in the config partition??) Or do the two interfaces share the same one?

I've discovered that my SOIC clip has a manufacturing defect and might not be making proper contact with one of the pins. So the next step will be to address that problem. But there may yet be hope for recovering something intelligible from the EEPROM of the bricked device.

@MK24

Thank you very much for all that information. I'm afraid I was busy typing my post (I'm embarrassingly slow these days) and didn't see yours until afterwards, so most of what I wrote is probably redundant.

It's way past midnight here, so best to get some sleep now. But thanks again.

There is no "config" partition in OpenWrt.  Factory firmware may use one but OpenWrt stores all the user's settings as various files in the jffs2 filesystem in the firmware partition.

You can't read all the partitions from one source but you can combine the files later.  Read "firmware" from /dev/mtd5, will be the entire kernel and filesystem space.
cat uboot.bin firmware.bin art.bin > wholeflash.bin
Or you could
cat /dev/mtd0 /dev/mtd5 /dev/mtd4 > wholeflash.bin
Doing either of these in place on the router will use quite a bit of RAM, which is scarce in the MR3420.

The wholeflash file should then be an exact representation of everything in the flash chip, unless some process was writing to the jffs at the time it was being read.  That is unlikely in a stock configuration.

(Last edited by mk24 on 1 Sep 2017, 02:25)

Thank you. I should have realised sooner that the 3.3V being fed into the EEPROM would also be finding its way to the SoC and potentially powering that up too.

With great difficulty I have managed to desolder the EEPROM chip from the board.

After mounting the bare chip in the clip Flashrom appeared to read it OK and the MD5 sum was different to the 'FF' dumps last night. When I ran Flashrom a second time to get a second dump for verification it wouldn't recognise the chip. Then disaster struck - leg #8 fell off.

Even worse, when I checked the first dump it was filled with '00's and the occasional block of 'FF's - so no usable data. Whether or not that is actually stored on the chip or just an erroneous result caused by a fractured leg on the chip is unknowable, unless by some miracle there is a way to repair the leg. But it has broken off flush with the plastic.

I had a replacement chip to solder back onto the board on order anyway, but it will take a while to arrive. In the meantime I can move on to the task of dumping the contents of the working TL-MR3040.

If anyone has any ideas how to bodge some sort of connection to the broken pin so I can try to read the EEPROM again, please let me know. But the situation looks pretty hopeless. :-(

You could try to solder a fine wire to the nub of the pin that is in the plastic.  Take a piece of stranded wire and unravel it to get fine wires. 

Get two 4-pin headers and make up all the pins on the chip that way with fine wire leadouts to the headers.  Then plug the headers to your programmer.

@mk24

Update
After a few days in bed, I've finally managed to sit down and work my way through your (superb!) instructions and have backed up the bootloader & art partitions from my working router. I haven't used scp before and spent the entire afternoon staring at error messages and alternating between Linux Mint & PuTTY on Windows. But eventually I stumbled across the correct syntax and succeeded with pscp.

Both binaries look OK. For good measure I made 3 dumps of each, rebooting the router in between, and confirmed that the hashes matched. The art binary contains a block of data from 1000-143F, but is otherwise FFs.

Is the MAC address stored in clear text or scrambled in some way? I searched the art binary for the working router's MAC address, but couldn't find it.

Next on the to do list: Make one final attempt to recover some data from the bricked router's EEPROM. Build a new EEPROM image from the dumped partitions & OpenWrt sysupgrade binary. And re-order some new EEPROMs. (The overseas vendor I ordered from a week ago has still not dispatched my order, so I'm going to cancel.)

I was unable to find the original Winbond EEPROM from a mainstream supplier here in the UK. RSOnline have similar parts from Cypress & Adesto.

Am I correct in thinking that so long as the package (SOIC-8) and basic specification (voltage, capacity, frequency etc.) are the same as the original part they could be used as a replacement? (Subject to Flashrom support.) And are there any pitfalls I should be aware of when checking the specification? Does the arrangement of the memory matter? (e.g. 8M x 4 bit, 16M x 2 bit.)

Thanks again for you help.

I have been using Winbond W25Q128FVSG.  Besides being one of the few 16MB chips in an 8 pin package, they will work with the older CC kernels if you ever wanted to run one.

The chips are pretty generic in operation but the kernel does look at the chip ID (JEDC) bytes and refuse to work with certain chips.  Newer builds include recognition of more chips.

Some chips have modes where more than one bit can be read per clock cycle.  These are not used by any router I know of.  Since the firmware is booted into RAM, the slower standard one bit mode is all that is used.

(Last edited by mk24 on 7 Sep 2017, 00:31)

Thank you. I don't want to be in the position of not knowing if something is faulty or just incompatible. So I shall stick with a Winbond W25Q***FV (even if it has to come from Ebay) rather than trying another manufacturer from an established supplier.

They used other EEPROMs on earlier revisions of the TL-MR3040. The OpenWrt TL-MR3040 page (presumably the v1) lists the flash chip as a Spansion S25FL032P (now discontinued) and the one stripped down by 'Jarocks' contained an Eon flash chip. But now is not the time for experimenting & making this job harder than it needs to be. (I took on this project because it looked fun & interesting.)

Ordered 2x W25Q64FV from France. (The UK seller had a reputation for selling used ICs as new. Don't think they'd bother for something as cheap as these EEPROMs, but it's the principle. So worth waiting a couple of extra days.)

Eventually I succeeded in soldering a fine copper wire to the base of the broken pin, after filing the plastic back slightly with a fine diamond bit. The join appears to be OK, but after connecting everything up to the Raspberry Pi Flashrom cannot recognise the chip. I wouldn't be surprised it was damaged by the heat while soldering, or maybe the join to the broken pin isn't successful.

The broken pin is number 8 (+3.3V). I tried measuring the voltage between it and other pins to determine whether or not the repair was successful. Initial results looked promising and I measured 3.3V between one of the other pins & ground. But after further investigation suggested my methodology was flawed. After disconnecting, I tried continuity testing between pin 8 & pin 4 (ground). The result was negative, but I'm not certain whether or not that indicates the repair to pin 8 is good or not.

Oh well, at least I'm having fun.

With no power applied to the chip, set your meter on "diode test" and test between pin 8 and ground in the reverse direction (red meter lead on ground pin).  It should show about 0.45 volts.  This also works for any other pins since the digital I/O lines have protective diodes to ground.

(Last edited by mk24 on 8 Sep 2017, 14:35)

(I keep forgetting that the diode function exists.)

Pin 8 (-) to ground (+) gives 0.32V. Also managed to get ~0.45V between some of the other pins & ground. However Flashrom still won't recognise the chip, so it looks like I finally killed it.

But I now know that that my attempt at soldering a wire to pin 8 was successful, so no need to try again. The new EEPROMs should arrive early next week.

[Also, nice know I didn't stab my myself with a hot soldering iron & pair of needle-pointed tweezers in vain. The former was a result of an involuntary spasm, and the latter a reflex action when the hand holding the tweezers came into contact with the hot iron! roll]

8MB/64Mbit binary file with Uboot & ART from the other TL-MR3040 and a downloaded sysupgrade image of OpenWRT 15.05.1 (ar71xx) prepared & ready. And now the W25Q64FV EEPROMs have arrived.

Next obstacle: Chip detected as a Winbond "W25Q64.V", but when trying to write the binary image Flashrom reports "ERASE FAILLED!". After letting it try 4 or 5 different erase commands it reported "No usable erase functions left".

Flashrom's successful detection rate is <100%, so there is probably a communication problem somewhere. I abandoned the clip and tried soldering the chip directly on to the pads on the board provided with the clip. And turned off any nearby sources of possible RF interference (even my cordless mouse). But no luck.

The cables connecting the breadboard to the Raspberry Pi  were salvaged from a PC case, connecting the front audio ports to the motherboard. They have nice terminals to connect to the Pi's GPIO header, but are not of the highest quality and longer than they need to be. (It was an astonishingly crappy case - it actually caught fire the first time I connected the front audio sockets up!)

I've sourced some nice short F->F jumper cables, which should arrive before the end of the week. Hopefully they might resolve the problem.

The EEPROM is now connected to the Pi via some nice short (~10cm) wires, but unfortunately the problem remains and I am still getting the "ERASE FAILED!" error.

After much searching it appears that while most people have been able to flash their Winbond W25QxxFV using Flashrom without any problems, while some others get the erase failed error. The cause of the problem seems to be unknown.

Any suggestions?

Hmmm.... Tried the second (untouched) W25Q64FV and had the same problem.

According to the datasheet write & erase functions are blocked when the VCC voltage is low. Checked the voltage with the EEPROM mounted and I get 3.326V between pins 4 & 8, which should be plenty.

Is it possible that these EEPROMs are a locked state where something needs to be done on pin 3 ('Write Protect Input' - not connect for SPI flash) to enable them to be erased?

Problem solved. After connecting pins 3 (WP) & 7 (HOLD) to 3.3V, Flashrom erased, wrote & verified the W25Q64FV successfully. big_smile

I put the EEPROM (containing the Uboot & ART from the working TL-MR3040 and an OpenWRT sysupgrade image) into the bricked TL-MR3040. But alas (as before) only the power LED comes on. sad

Looks like the next step is to see if the serial console on the bricked router is now working.
If not then I'll have to remove the EEPROM and read it to double check that it wrote successfully. Then go through my ROM image again to see if I made any obvious errors.

EDIT
My pessimism was premature. Forgot that this would be a fresh OpenWRT installation, not the DSLRContoller image I used before. Can you guess where this is going? roll

That's right, the Wifi was switched off in OpenWRT by default. And there was nothing plugged into either the USB or Ethernet ports, so their LEDs were off as well. A few minutes after originally writing this post I came to my senses and thought I'd better try plugging in an Ethernet cable. Sure enough the LED immediately came on and I could log in to the router on 192.168.1.1. After turning the Wifi on in OpenWRT that LED came on also and my laptop can see its SSID. Eureka!

Thank you so much for all your help mk24. cool

(Last edited by davelister on 17 Sep 2017, 03:13)

The discussion might have continued from here.