I first heard this story as part of Kwabena’s presentation on OpenMV, a camera board.  He spoke at the Oct 2016 Open Source Hardware Summit in Portland Oregon. As he described the path from idea to product I had an inkling this would be a good story for The Engineers’ Daughter. –Anne Meixner

Ibrahim dreamed of building a better camera board that could be made available to other hackers. As he dreamed he blogged. With interest from the community he brought the camera to Hackaday; this brought the dream to a wider audience. Then came the push to make it a reality with a successful Kickstarter program- achieving double the fundraising goal. This is when I became involved. I’m here to tell you that the road to making this a reality had a big pot hole in which a difference in ovens played a subtle role.

As we chose the parts for the board we made a pivotal decision to use an image sensor from Omnivison, OV2460. The chip had multiple attractions. Built in JPEG compression for one. This enabled higher resolution pictures and video which could be stored on the microcontroller’s RAM. The price was quite nice- $2. It did have one nagging characteristic- age- a 2006 design. Naturally, these BGA packaged parts had been sitting around for quite the while. Well our prototype boards worked well. If our hand builds worked, what could go wrong at the manufacturers?A difference in ovens and old BGA parts combined to bring us 80% failed boards. Yikes- good thing we tested these boards before shipping to our backers. The root cause of the disastrous yield—a poor electrical connection. Solder balls can microscopically rust and corrode, invisible to the human eye. During the reflow process they don’t successfully melt into the board’s solder paste. The defect is commonly called the head-in-pillow effect. Although the solder ball physically bonds it does not electrically bond.

Why didn’t this happen with our prototypes built by hand? Well we had hotter “ovens”. In actuality, we had a heat gun and we just kept applying the heat gun until it worked. We blindly assumed that manufacturing wouldn’t have an issue. Mass production involves automated machines and prescriptive processes, it’s one pass through the reflow oven. Our manufacturer was not at fault. Our lack of experience with the production environment and using 2006 vintage parts created the situation.

We needed to engineer our recovery. Recovery with the existing design did not seem possible. What kept us going was our partnership with our contract manufacturer- MacroFab and the transparency that we maintained with our community of backers.

Research by Macrofab uncovered that Omnivision never sold the OV2460 freely on the market and production had long ago stopped (recall this was a 2006 design). The chips we had were either reclaimed from recycled product or were spare stock from the 2006-2007 timeframe.  Unlikely that the chips were stored properly all those years to prevent the microscopic degradation.  Lesson learned-  know the source of your parts.

We examined what we had on hand: $18,000 reserved for shipping the rewards, only 20 camera modules working, 1,450 bad camera chips and PCB boards. Naturally, we also had some 900 disappointed backers. The community enabled making Ibrahim’s dream a reality; it was now their dream as well. We admitted to the current nightmare and communicated that we had enough funds to do a new design and manufacturing.  If the design worked, we would need their help to ship the parts.

The redesigned module used Omnivision OV775 which unfortunately did not have the JPEG support – we lost the higher resolution and video.  Some backers requested a refund which we willingly gave. We built 10 prototypes which all worked! We ordered parts to do a small manufactured build of 40 Open MV Cams.  By limiting our order, we could recover if we ran into issues. We had 100% working modules- so we went into full production with confidence of no manufacturing issues. In parallel, we worked evenings and weekends on the software/firmware for our camera.  We added features and crafted improved firmware. All along we kept our community updated with detailed reports every two weeks. The remaining backers raised the funds to permit shipping OpenMV Cams.

On making a dream into a reality, Ibrahim and I learned some key things due to the potholes in the road. The maker revolution relies upon community, on sharing knowledge, on helping each other.  Ibrahim’s search for a better camera resonated with others, so the camera was built to enable future maker projects. From prototype to a small manufacturing run, we made some rookie mistakes. It is in the doing that we learn. Mistakes happen. If you choose to give up, then you do not learn. As part of this community I have shared what we learned.

Have a productive day,

Kwabena W. Agyeman

Dear Reader, What memory or question does this piece spark in you?  Do you have some experiences with prototypes and manufactured parts having differences. Please share your comments or stories below. You too can write for the Engineers’ Daughter- See Contribute for more Information.

Additional Reading

To learn more bout the Open source Hardware movement start here:  https://www.oshwa.org/

Key learnings from this project include:

• Verify your manufacturing process prior to buying all your parts
• Take intermediate steps when checking your project.
• Be savvy on sourcing your parts- we went cheap & old and lacked historical knowledge.
• Be persistent, you’re going to hit some rough patches,
• Community Community Community. They made the dream a reality and we owed it to them to keep them informed.  If we had kept our mouths shut and not let them know of the issues, then we would have been another failed project.

 

The blog that started it all July 2013.

 

Fred Schenkelberg, a reliability consultant, shared some useful tips on reliability and prototypes.

Scroll down The 2016 Open Source Summit recap to hear the presentation.  You can download his slides here. Interested in openMV- go check out their site.

http://hackaday.com/2015/11/05/fail-of-the-week-openmv-kickstarter-project-hits-manufacturing-snag/

https://www.kickstarter.com/projects/botthoughts/openmv-cam-embedded-machine-vision

https://macrofab.com/work/openmv-cam/