Dear ON Semiconductor
We unleash the professionals at Atmel to inspect the insides of our defunct ATmega328s. We found ON Semi written inside. Tim gives us a tour of the Failure Analysis labs at Atmel, Colorado Springs campus.
Dear ON Semiconductor,
I'm afraid I have a reel of your ICs, but they have been mistakenly mislabeled. Could you please let us know what ICs these are? Here is the wafer identifier:
Marked with a label 'AG20 01 - 7'. And if it helps, this is what the entire wafer looks like:
These dies are encapsulated into a TQFP 32-pin package. We'd really like to find out just for curiosity's sake. Please email spark@sparkfun.com. Thanks!
Sincerely,
Nathan Seidle
If you've been following the saga of the fake Atmel ICs, you know what's going on. If not, read back in time a bit: ATmega Slugs and X-Rays. So a few weeks ago, our writer Chris walks into my office and against all reasonable odds of coincidence in this world, Chris lets me know that his very own father (Tim) works in the Failure Analysis labs at Atmel. What? Really? Where? In Colorado Springs about 2 hours away. You're kidding?! Tim had kindly offered to have us come through the lab and analyze the ICs for us.
Atmel is a large, international company with a decently sized wafer fabrication facility in Colorado Springs. There's a good chance that some of the very ATmegas we use on our boards could have been fabricated there! The wafer fabrication area sometimes experiences processing or testing failures. The failure analysis group is there to evaluate problems and discern how the problems were caused. But their 2nd job is to help customers when a part fails in the field. Counterfeiting is extremely rare, but not unheard of. Luckily, the processes to do fault analysis are similar to counterfeit analysis - AKA popping the top off and looking inside.
So a contingent of SparkFun geeks headed down to Colorado Springs on a road trip with a reel of ICs under our arm.
We met Tim in the lab. He showed us the various tools they use for analysis: microscopes, scanning electron microscopes, a laser to heat a single transistor junction, toys, toys, and more toys. We were in awe.
The SEM with probes is capable of taking a reading from a single NPN transistor junction.
A close up of the probes. Each probe head had a rough tip radius of 0.25 microns. That...is small.
"Xena" and "Hercules" are the two SEMs in use. They have the ability to scan down to around 10nm. Check out the monitor - this SEM is set to accelerate the electrons toward the sample with 2,000 volts!
After the much-too-short tour, Tim threw on his protective gear and started dissecting the "ATmega328."
Here's what we learned: 90% fuming nitric acid at 150C is the key. The "fuming" part of the nitric acid makes it attack organic material (namely the packaging) while leaving the inorganic material (the IC chip itself, including the bonding wires). We believe our first attempts at decapsulation went so poorly because we used concentrated nitric acid instead of this fuming nitric. This stuff is not cheap! ~$250 for 500mL, but you really only need a few drops. Heat the IC on an empty glass petri dish. Add one drop of fuming nitric acid. Let it sit for 4 or 5 seconds, then wash off with acetone. Don't try this at home! This combination of acid and a solvent is potentially very dangerous. We let the experts handle it, but really - not a good idea for the home chemist!
The small drop of acid quickly devours anything near by and then becomes used up. To wash the nitric acid away, you could use water. But combining fuming nitric acid with water, it turns into regular nitric acid that then begins to eat away at the inorganic materials (like the silicon we're trying to protect!). So Tim uses acetone to quickly neutralize the acid. Now for those astute readers/chemists, you will know that combining a strong acid with a solvent, you get a very exothermic reaction. This is dangerous and should not be attempted outside of a laboratory setting. Very small amounts of nitric acid are used to minimize the exothermic reaction to maintain safety. Again - don't try this at home.
Wash, heat, add drop of acid, repeat. This process took only 2 to 3 minutes. Tim had obviously done this before.
CMac (Chris on the left) and Nate (me on the right) hang back and watch the fun from a safe distance.
After a few short minutes, Tim had the top of the IC expertly removed. He then added the IC into a small bath of "NMP" (another smelly solvent) and then into the ultrasonic cleaner for 10 to 15 seconds. The capsulating materials contain small glass beads that enjoy sticking around. This cleaning knocked lose any residual fragments of the packaging that may have been sticking to the chip or bonding wires.
We then got to view the uncovered IC under a microscope. We immediately saw the ON SEMI markings that re-affirmed our suspicions. There's something in there, but it's not an ATmega328.
There's the give-away marking from ON SEMI. Now if you know anyone at ON Semiconductor, we'd really like to hear from them. It's going to be tricky to identify these without their help. AG20 is quite clear, but 01-7 could be anything. After talking with the folks at Atmel, each company has their own very proprietary naming nomenclature for the wafers. Who needs to know it anyway? No one ever sees anything but the label on the outside of the package, right?
The various failure analysis detectives within the office commented on how old the technology looked. They estimated the wafer to be between 5 and 10 years old. ON Semiconductor was started in 1999 so they can't be much older than that.
According to the package markings on the right, the fake ICs were produced in the 23rd week all the way back in 2007. To put the date code question to bed, we also received some information from the production facility that first produced the ATmega328s. This is what they said:
The ATMega328 was first taped out toward the beginning of 2007. Following an evaluation phase, it went into volume production toward the end of that year. It is very unlikely that there were any parts from earlier in 2007.
The date codes for December 2007 product should have been something like 0748, 0749, up to 0752. I think the date code on the counterfeit parts was 0723 which means fiscal week 23. Sounds like this would be prior to when this part went into any type of volume production, just like the test guys suspected.
So it turns out we really did receive counterfeit ICs (we decapsulated three ICs with identical results). It's not hard to see how this could happen. The going rate for the ATmega328 TQFP is somewhere between 2 and 3 dollars each. Someone had some extra/scrap ICs sitting around in the same TQFP package. Every IC gets a laser etched identifier, in this case it was just the wrong label, (intentionally wrong in order to defraud us, but the laser doesn't care), and wammo! You've got your very own ATmega328P-AUs.
Thank you Tim and all the folks at Atmel! The tour of the wafer fab and the huge campus that Atmel has down in the Springs was really amazing to see! Please come to SparkFun some day so we can show you your very own ATmegas in use.
If you read this post and find anything out to help us identify these ICs from ON Semiconductor, please let us know! We're still hoping to turn one on someday, whatever it may be...
Be sure to read the original post, the follow up report, the discovery of what's inside and the final identification of the IC!
Wow - what a fascinating story! No doubt you could turn this into a best-selling detective novel for geeks. And what an amazing coincendence that Atmel connection was.
I'm also amazed at how clearly the photo of the die turned out after going through the acid fuming process. You dont know how tempted I am right now to try reproducing this process in my basement. Dont worry though - I'll heed your warning about trying this at home :)
LOL
(yet another spammy, meaningless comment)
Are you talking about his comment or yours? I don't seem anything spammy about either one!
Mine
Aren't you infringing or helping infringe copyrights by displaying the inners of that chip?
I'm pretty sure that putting up pictures of a product doesn't violate any copyrights...
-Taylor
Alright, was just wondering
@ARMinator: Even if so, this isn't anything that their competitors can't do in an equal amount of time (Since, after all, they just did.)
IANAL, but I'm pretty sure this would fall under the "Hardware Interoperability" clause of the DMCA
What a mystery! This is more exciting than any C.S.I. television show could ever be. I hope someone out there can help solve this. A great story like this needs an ending.
Thanks to Nate for writing up another excellent chapter in "The Case of the Missing Microprocessors."
...and here all along I though "fuming" nitric acid was just another angry acid.
Richard Hart: What a mystery! ...
...and here all along I though "fuming" nitric acid was just another angry acid.
From the description, I STILL consider it an "angry acid". The worst kind. :>
That part is indeed an NCP5318.
http://www.onsemi.com/PowerSolutions/product.do?id=NCP5318
The AG20 represents the internal die number for manufacturing and data management. The 01 represents the metal mask version (i.e. metal masks to create a family of parts from a base die.) And the -7 represents the die revision (i.e. it took 8 tries to get it right.)
How or why this part found it's way into that package is hard to say. Most likely ON and Atmel use the same package vendor. They messed up at marking. Either 'scrapped' it at the site or sent it to Atmel. Atmel 'scraped' it when the whole lot failed. (Check for marks on the leads to determine if it when through a handler for testing.) At either of those scrap points someone probably went dumpster diving.
Part II
As far as the marking on the die, I dont think that those are date codes. They are identifiers for the die. Some companies sell raw die and they publish the die code in their data sheets. Date codes are for the customer. The manufactures track by lot numbers. Some more expensive dies have lot numbers laser etched or serial numbers programmed at test. On the factory floor they are very careful to keep the lot traveler with the parts. If they get mixed up (unlikely) they have test programs to determine what the part is.
I live in Taiwan so I dont want to say that anyone would do that here. :-)
Counterfeit shops are all over, even Taiwan unfortunately. Some use older packaging equipment which is probably used and resold equipment. Others are made with state of the art equipment which indicates that some high end contract shops may have "rogue employees."
Part I
I worked for TI as an automation engineer lets just say for a very long time.
The parts are laser symbolized. This is an expensive large machine. A mold press and tooling for packages is also a large and expensive machine. Not something that a garage shop would have.
These were probably done at a manufacture of semiconductors. If I remember correctly the mold mark showed Taiwan. There is a ON Semiconductor site in Taiwan. The lead form looks good so they were not just dumped in the trash. Also you cannot resymbolize laser markings. The plastic is quite hard so it takes a lot of power to etch the plastic. Someone could have acquired the parts before they were symbolized and had them symbolized in China. If I remember correctly the symbolization was done after test at TI and I would assume other manufactures do the same to not waste time symbolizing bad parts.
Just a couple of counterpoints based on experience:
I was talking about the die being salvaged from scrap. The die are often at least partially tested prior to packaging. Whole wafers may be scrapped due to results of those tests. Normally they are destroyed, but not if they are stolen first. This is not hypothetical.
Laser markings can be remarked. The top surface is sanded (blacktopped) and then remarked. http://www.aviationtoday.com/av/issue/columns/1078.html It is noticeable when you see it though. In this case that doesn't seem to have happened
This is one of the reasons I find Sparkfun one of the best electronic sites out there. It is just the extra effort that they put into things like this that makes me buying expensive hobby electronics with a big smile, knowing that I am giving my money to people who will do anything to satisfy their customers.
"The failure analysis group is there to evaluate problems and discern how the problems where caused."
'Where' should be 'were.'
this saga has been the most interesting and enthralling read i've had in a long time. to turn lemons to lemonade, i'd say this whole ordeal provided a story that makes up for [at least some] of the headaches it's caused in designs and projects. then again, i wasn't one of the ones who sat ripping their hair out wondering why my grand design wasn't working c/o a left-field problem like counterfeited chip (i could only imagine how frustrating that could be, as i personally wouldn't have that possibility in my troubleshooting process).
can't wait for the next chapter to unfold
When Motorola spun off their semiconductor business, Freescale got the LSI circuits while ON Semiconductor got the discrete devices and MSI circuits so this is not a processor. The large metal chunks (the white areas) on the chip are probably capacitors so this may be at least a semi-analog circuit, maybe a power management circuit or something similar.
It is interesting there are 32 pins on the package but 33 bond pads on the chip. The extra pad is the one in the center on the right side and looks like the VSS pin since it is connected to the metal ring around the periphery of the chip to which all the ESD protections are connected. Have yo figured out where this pin goes?
I hope this can get posted on Bunnie's Blog. The readers there are pretty amazing when it comes to recognizing layouts, they could probably give some clues even if ON Semi isn't as cool as Atmel.
Seconded. If anybody can figure this out (aside from ON), it's Bunnie and his readers. Every month I'm astonished at how quickly they solve his hardware identification puzzles.
Splendid story, I wish I bought one while I could, by now I'll probably have to bid 50 times an original 328 to get one ;)<br />
<br />
But.... can't you just tell the last guy hired to peal layer after layer, take pictures, trace components/routes/netlists etc and.... reverse engineer the chip ?<br />
<br />
If he succeeds the story will unfortunately end, but... you'll know he's willing and capable !<br />
Possibly someone made a mistake in laser marking at the assembly site and then someone mistakenly thought they were as labled and sold the reel as Atmel parts. All sorts of crazy things go on out there. Always buy from valid distributors. Otherwise you get what you get.
Maybe I missed this, but are these truly counterfeit? i.e., do they somewhat function the same as the genuine ATmegas?
Or are they some other die in a faked package?
There are many kinds of counterfeit, but this is one kind. They were sold as to sparkfun as Atmel parts and they are not.
Other counterfeits types include empty packages, marginally failing devices, or devices marked to indicate that they have expensive testing done on them when it has not been done. In all cases fraud is involved.
It's serious issue and there are confernces on this topic (for example http://www.smta.org/counterfeit/)
Check out the past homepage posts from us. They are whatever they are, but are not what their label indicates. Call them counterfeit, fake, blunders, mis-prints, etc. But they aren't what they say they are.
Awsome article and pictures!
This story is another Cuckoo's Egg...
http://www.amazon.com/Cuckoos-Egg-Tracking-Computer-Espionage/dp/1416507787/
These ATMegas probably track back to a guy in Germany who is involved in drugs and the KGB... you just wait.
Talking with an ex-ON employee he noted that when ON seperated from Motorola they did not re-marked their parts like that. The M had 'bat-wings' and they would have left the part marked as Motorola. He also said they never coded their parts like that either. It might be a fake within a fake... although if the '01 - 7' was a date code (2001 - week 7) the week code would have been a 2 digit code (0107). He remembers many shipments of parts which were lost and stolen during that time. Who was the distributor?
The (M) represents Maskworks. Just like (C) is for copyright. The (ON) is the equivalent to the Motorola M, but for the new company.
@dropD: I don't think that Sparkfun called the distributor out on it; from what I heard, they just stopped purchasing from them.
This is a geek's wet dream. Well, I enjoyed the story anyway!
Now, about the nitric acid. You can make your own fuming HNO3 by buying concentrated acid (more on this below) and... just add water! Nitric acid 'fumes' by releasing nitrous oxide when there are enough H+ ions present. I can't remember the exact concentration to make it fume, but I think it's about 60% or so.
Now, about the cost. The figure you were given is almost certainly for "Aristar" (ultra-pure) quality reagent. You can get cheaper "Analar" (analytic quality) for about 80% LESS. And you can buy gallons/litres of less pure acid from just about any quality chemical company. Just say you want "standard" (non-analytic) acid. I can buy 4 litres here in Oz for about $75. The same quantity of ultrapure reagent is over $1200!!!
I hope this helps.
-PC Pete
holy crap you serious??? I would never have dreamed about purchasing Nitric acid in Australia... You can't even buy acetone these days let alone nitric acid! :)
Microchip and ON Semi tried about 2 years ago to acquire Atmel. Attempt of decreasing Atmel's credibility?
Fab (sorry) story! Written up here, and I'll be watching avidly to see how things develop...
http://www.zdnet.co.uk/blogs/mixed-signals-10000051/fuming-nitric-acid-robots-and-fraud-chasing-counterfeit-chips-10017853/
Rupert
Fantastic writeup! Thanks!
'no-good naughty gubbins'... Ah, I so love the British!
Here's my 10 cents worth...
If you're gonna fake something, make it worthwhile for yourself. So, pick an existing chip with the lowest value, right?
How's this for a candidate...
http://search.digikey.com/scripts/DkSearch/dksus.dll?Detail&name=NCV7513AFTR2G-ND
It comes in at $1.92 per unit, which results in at least a $1 profit on each chip for the counterfeit version.
Simmers
Superb detective work.
I'm no expert at silicon chips, but just going by the layout I see. I suspect this could be your part.
http://datasheet.octopart.com/MC100EP451FAG-ON-Semiconductor-datasheet-623768.pdf
Of course, I could be way off the mark too.
Am I the only one that things AG20 01-7 looks like a date code? Either July 2001 or 20th Jan 2007?
Glad to see the mystery solved! Thanks for the detailed writeup!
That SEM is made by my company, JEOL. I however do not work on SEM's, I'm an NMR guy so I don't think I would be much help...
Nice photos of the setup. Unfortunately this incident is not uncommon in the industry with all the counterfeit parts out there. I doubt most developers who get them do failure analysis. Be careful who you buy from.
Just one nitpick with your letter, the photo is of the die. There is a die identifier AG20 and then probably a date code that shows what lot the wafer came from. Lots of die are cut from the large circular wafer.
It's likely these got stolen from a low yield lot that was in the scrap bin.
There seem to be 33 bonding wires on the die. I wonder where the extra wire goes...
That wire connects to the blue smoke container underneath the IC.
Hi,
Just a wild guess could be the NCP5331 from ON-Semi. The NCP5331 is a "Two Phase CPU Controller with Integrated Gate Drivers for AMD's Athlon Processor" and is packaged in a TQFP package. It has 4 gate drivers and it look like there is 4 similar circuits in the upper left corner on the die. The datasheet date is March 2005
It's the wrong package (LQFP-32), but at least some of the pins seem to fit if you turn the chip 180 degrees.
http://www.onsemi.com/pub_link/Collateral/NCP5318-D.PDF
Hi Zach - you were correct! It is the NCP5318.
http://www.sparkfun.com/commerce/news.php?id=395
Nice work!
I'm just a simple Mech Eng who dabbles in controls, so take what I think with a grain of salt...I see 4 identical circuits in the upper left corner, with a common connect at 'pin 1.' 'Pin 5' looks like a common rail - Vcc or Vss. Same with that extra pin to nowhere - I would suspect that's a ground pin into a shield or something under the die.
I would guess this is a type of comparator or quad switch/gate/logic device - probably high volume, low cost (note alot of ON's switches/gates are $0.11 - $0.25 in bulk!) Good way to make bucks, if you can sell them as $2-3 MCU's!
"Zena" and "Hercules" are the two SEMs in use.
You misspelled the warrior princess! It should be "Xena" :)
Fixed! 1995 was just so long ago!
This makes more sense than the original copper slug theory. Very interesting, thanks for sharing the story as it has unfolded!
This may be the chip.
http://www.onsemi.com/pub_link/Collateral/MC100LVE164-D.PDF
running off to test
Try to test as this one too:
http://www.onsemi.com/pub_link/Collateral/MC10EP131-D.PDF
I doubt it. The MC100LVE164 has 7 NC pins. It doesn't look like the fake IC has any NCs.
Your thougts gave me idea.
As on images above 3 areas repeated four times (as for my untrained eye). When looking on this list (QFP keyword search results), it should be "Quad something".
This join gives us only 6 part numbers, and only 2 of them has no NCs, they shares same datasheet:
http://www.onsemi.com/pub_link/Collateral/MC10EP131-D.PDF
How do you figure that?
My untrained eye would expect to see more symmetry and repetition in a 16:1 multiplexer, plus these images look way too complex for that.
You can contact them here:
http://www.onsemi.com/PowerSolutions/ticrequest.do
"Wow" pretty much sums it up for me. I was blown away by the shot of the half-eaten 328, how the casing was removed without so much as a scuff mark on the bonding wires. Very cool.
Nice work, and a great story. I can't wait to see what these finally turn out to be!