The Birth of castAR: Q&A with Jeri Ellsworth and Rick Johnson

We sat down with engineering superheroes Jeri and Rick at this year's Consumer Electronics Show to talk about castAR, Kickstarter and the value of accidental discoveries!

by Chelsea the Destroyer April 21, 2014 4:41 pm UTC 3

• Q&A

Favorited Favorite 0

At this year's Consumer Electronics Show, SparkFun's Technical Researcher Pearce and Director of Marketing Jess were fortunate enough to nail down electronics superstars Jeri Ellsworth and Rick Johnson to talk to them about their company Technical Illusions, the castAR glasses project and taking Kickstarter by storm. A lot of us at SparkFun look up to Jeri and Rick as personal tech heroes, and we were grateful that they took the time to sit down and chat with us about their triumphs and challenges. Let's check out what they had to say!

Pearce: What is castAR?

Rick: CastAR is a pair of glasses you put on that projects out a holographic, stereoscopic-like image that you can interact with and move around, and visualize games, data and collaborations. It has a clip on that allows you to have a full VR experience, as well as an AR experience. We have two input devices: a magic wand, which allows you to interact in a 3D environmental way to the scene, as well as an RFID grid which tracks any object that has an RFID tag. So you can imagine board games scenarios - D&D figures, Magic the Gathering cards, anything like that - can be tracked across the surface, augmented with data or have the computer respond to it. You could put down a Magic card and a dragon could spawn from it; a D&D figure could have stats; you could put down a physical miniature and then somebody playing remotely could see a virtual miniature, and you can share board game environments now remotely. In addition, we have an advanced version that uses a microprocessor, and this allows two-way communication between the grid and the microprocessor, so you can have switches, control motors, smoke generators, lights – the idea I like to use is a virtual race car, or a physical race car controlled by the computer, driving around a virtual racetrack.

Pearce: You did a Kickstarter – how did you feel about it? In the past we’ve seen Pebble and others that make an overwhelming amount of money; do you think what you raised was overwhelming?

Jeri: I was very happy with it. We hit the dollar figure that we wanted to hit. It was pretty exciting; there were only 11 Kickstarter campaigns that got over a million dollars this year. We had a lot of anxiety going into it, because it’s such a complicated thing to express in video or text, or describe in words to folks until you actually try it, and we weren’t sure how people would receive it. So starting at Maker Faire in spring 2013 (that was our first release), we started showing it in public every couple weeks or monthly clear up until Kickstarter. We showed thousands of people, so there were many people out there that would be there to say, “It’s for real; it does what they say it’s going to do.” And we tried in the Kickstarter video to shoot through the glasses even though it doesn’t look as good. We could have rendered it and maybe made it look snazzier, but we just wanted to be very honest.

Rick: Jeri and I were engineers at heart, and everybody will tell you when you run a Kickstarter campaign that it’s completely disruptive to your life, and that’s 100% true. Pretty much from a month before Kickstarter and through the campaign, we never did any engineering to speak of. You have to come up with a script for the film, content for the film, you have to make the film, edit the film, create text for the Kickstarter page, set the financials for the rewards, write the reward text, make sure Kickstarter’s happy with all of it, hit the launch button, and then you’re doing interviews and additional content, and stress and stress and stress, and then you’re done, and then you can think about doing engineering work again.

Jeri: We were going to do a couple videos a week, but it actually turned out to be maybe one-ish – we could barely get enough time to do another update video.

Pearce: I’m in technical research; I have to go out and find the parts for us. Our CEO Nate comes back with some wild idea and I have to go find the parts for it. How difficult did you find it sourcing the components for the castAR?

Jeri: Extremely difficult. And that’s always the case in every startup, so it’s not unique to what we did at all. All the sales folks want to deal with the Sonys and the Microsofts and people who are going to want to buy 10 million units, and they look at anyone who’s a startup as a drain on their resources, so they’ll either never give you the parts, or won’t give you the data sheet. It took a lot of persuading to get the micro-displays and the image sensors and the various parts that we needed.

Pearce: Would you say that’s been the most difficult step of the process so far?

Rick: I think probably overall it’s the sheer volume of work that Jeri and I put in for the majority of last year. It was her and I putting in five to six days a week of 14- or 16-hour days, and prior to that point in my life I always worked in an environment where I had a ton of colleagues, and if a problem came up, I had people I could talk to. In this environment, I’m on the software side, Jeri’s on the hardware side, and there’s nobody else to help us with problems, so one way or another, one of us has to solve everything. It’s a daunting environment and it adds stress, and there were times when we were perpetually sick with colds for months at a time, but we persevered, and it was very rewarding.

Jeri: It was constant micro-adjustments throughout development, and we were always looking at three different directions we could go at any given point, so we theorized on how we could do the tracking, or how we could do the display driving, we were always ready to jump to another path when one got too difficult, so that’s how we got to the end product; there were a lot of jumps back and forth.

Pearce: I noticed you guys are doing game jams – can you explain that a little bit?

Rick: If you think about this technology – VR and all that – if I describe it to you or show you pictures of it, that’s a very different experience than when you just put it on and try it out, and we’re still at a point where the prototypes are hand-built and there’s only a few of them (and we cuddle with them at night so nobody steals them). So game jams are a way to help other developers gain an idea of the direction of gaming and what this technology is; we can invite people into our work environment and they can experience the tech firsthand and can try out their ideas.

The thing we learned early on, especially in the AR space, is that we don’t actually know anything about the AR space: how the user interacts with the space, how they see their score, how other people interact with it, et cetera. For instance, if I share the same physical space with another player, am I gonna be sharing my arm space with them? Am I gonna be pushing them around? Aspects like that you really just need to experiment with, and game jams are a way that people can naturally see that. We can provide them with information that we’ve learned, and they can come up with new ideas. It’s a way to explore this area before we actually have dev kits in the public space.

Jeri: It’s super fun, it’s a blast hanging out – it’s a good reason to get a bunch of fun people together, too.

Pearce: I know you’re working at getting the dev kits together, but what would you say your next steps are?

Rick: One of the bigger steps is that while Kickstarter has provided us with the funding to create the product for Kickstarter, we want to be a company that is sustained and has long term goals well beyond that. Initial steps are finding a businessperson to help grow the company, to find additional funds to grow the company, and hire staff so Jeri and I don’t have to sacrifice our lives in order to do this.

The secondary aspect is actually delivering on Kickstarter. There was a transition between the SD and the HD; Jerri has eliminated the control box, we’ve gotten the new projector engines up and running, and she has other technology that she’s advancing at the same time, so it’s taking all these steps to get us to delivering each of the Kickstarter goals. There’s other technology and things that we’re working on behind the scenes toward that, but it’s not stuff that we’re quite divulging at this point.

Pearce: A lot of our customers are looking to take the same path as you guys; they want to put together something to bring to market – maybe through Kickstarter or other platforms. Is there any single piece of advice you wish you had going into this?

Jeri: That’s a complicated question. It could be anything as far as international shipping and the weird psychology people have around paying more for international shipping than domestic, which was a surprise to us. We put a lot of thought into Kickstarter and I think the piece of advice to anyone going into it is: Don’t rush it. Take your time, think about the impact of big events that might be happening around that time. We strategically did our Kickstarter before Thanksgiving so the holidays wouldn’t get in the way, we didn’t do it in the summer when people were off on vacation, we made sure that we seeded the market by showing a lot of people the prototype, we talked to a lot of media ahead of time, and we had a really solid demo when we started talking to folks, so it was really plan, plan, plan. You have to also plan for post-Kickstarter too, and that’s the phase we’re in now – going through these different milestones, and miniaturizing things, and hitting these checkpoints that we plotted out ahead of time.

Rick: I would say that if you’re going to go into this endeavor and you’re going to be doing it with multiple people, find people that complement your skill set and are like-minded to you. Jeri and I are both extremely passionate about what we do, we both put long hours in, and we’re both very dedicated to it ... we both own Commodore 64s. You don’t want to find out too late in the process that somebody doesn’t have the will or the same energy level as you, or you have these weird divides on how to approach things, because you’re both in it for the long haul.

Jess: Were you surprised by the huge outpouring of support that the Kickstarter got?

Rick: There was a moment during the first Maker Faire where the day opened and we had people there, and we put a lot of effort into setting up the booth, and we had no idea what to expect. And the first person trickled in, and they invited friends, and all of a sudden we had an hour-long line, and then we had an hour-long line all day. Going through the amount of hours and stress we went through to get to that point, you have no idea what your confidence level’s going to be at that point. That evening we went out to dinner, and took everybody who came down to help us – it was about 12 people – and we got up and gave a speech to thank everybody who came, and that is probably one of the proudest moments in my life. All these people who believed in you and helped you out - it’s not what we did, but what WE did as an entire group - and it’s something that will always resonate to us a lot.

And when we went into Kickstarter, we decided to launch at 8:00 a.m Eastern time Monday morning, so the people filing into work who are bored have something to spend their money on. So we’re up at 5:00 a.m Pacific time ready to hit the launch button, and we hit the button thinking, “Is there going to be a backer at all?” And a backer came in.

I had inadvertently set the Kickstarter emails to come to my email account, and any time there’s a pledge, or a comment, or a change in a pledge, or anything, I got an email. And my email exploded, and I had the Kickstarter app and you get a ding, and ding-ding-ding-ding was happening. We went out to dinner at Chipotle, and I looked at it before we started in line, and then we sat down and ate dinner and I looked at it again and said, “Oh, we just made $13,000, I should just eat burritos the whole time.”

We went into Kickstarter with the advice we were given, which was that if we hit 25% or 30% in the first few days, we would probably hit our $400,000 target, and we hit our goal in a couple days. It was exciting, but we were so drained mentally; we were driving to another interview and we got a notification that we hit our goal, and it was like, “Ohh, yaaay…”

Jeri: I remember we were driving across the Oakland Bridge and all three of us in the car were like, “Yaay…,” and at that point we’d already done three interviews each day for the first two days.

Rick: And Jeri had been sick for a month including that time. The oddest thing that happened right at the end of the Kickstarter is that about five hours before it ended I developed this weird...almost like if I’d been drinking for two days – that nausea type feeling – and it lasted about 36 hours. And I almost think it was my body saying, “You’ve abused me for hours at a time with stress for the past nine months, and I’m done with that.” And you developed the same thing like a day later.

Jeri: I’ve been through a lot of stressful projects, and this one by far – the Kickstarter process – was the most stressful. The amount of PR that goes into it is incredible.

Jess: Can you talk a little bit about your prototyping process?

Jeri: If we go all the way back to Valve software, where the first concept came about, it was an accident that we ran into this property of the retroflective screen. I was working on an optics bench trying to solve issues with near-to-eye displays – that’s where the image goes directly into your eyes – and so I had this projector system with lenses and beam-splitters and stuff, and I was looking into it and accidentally put a beam-splitter in backwards, and instead of projecting into my eye it was projecting into the room. I had all these lenses and I was kind of fiddling with them, thinking, “Why don’t I see anything?” And I hit the focus just right so that it focused across the room onto this piece of material that we had there for another experiment and I saw this beautiful image pop onto the surface and thought, “Well that’s very interesting.” So I went and grabbed the material and I started looking at it, so part of prototyping and developing things is always being open and curious about things and making observations, and you never know when those things will lead you down a really good path.

I suspect that this is going to be a very comfortable experience; it solves some of these near-to-eye issues because the display is actually at a comfortable distance. The way I prototyped it was to put giant projectors on a hat – it was this giant hardhat type thing, we called it the "headcrab," which was one of the things in [Valve’s] games - and project it out that way. And we proved that we could do a stereoscopic image; I saw a 3-D image and thought, “Good, ok, this is one step closer to being able to have holograms on the table.” I just had this dream of holograms on the table, so we took a webcam and stuck it onto the headcrab, and we did a very crude head-tracking thing, but at first it was really terrible and not responsive at all. So it was these baby steps, and at that point I had a lot of pressure on me to prove that these big projectors could be miniaturized. I actually – for the company – had to go through these hoops I wouldn’t have normally had to, so I intentionally took a sidestep and miniaturized one projector down and made a pair of glasses with one miniature projector on it, just to show the viability of having a small projector. I didn’t know anything about optics - I didn’t have any optics resources, and I wasn’t allowed to get any - so I read a lot of books, I did at a lot of experiments, and I ended up building my own projector just by trial and error, and learned a lot along the way.

And then we left Valve. Up to that point we’d been working on our own hardware tracking system but it wasn’t working when we left Valve. We knew that was the key to our system, so Rick and I hyper-focused for a month on the tracking system and got our first version going. It was kind of funny, I put it together, handed it to Rick to test the software, and he said something like, “This thing’s broken –..."

Rick: The way I had it set up was that I had the camera fixed on my desk, and I had the tracking fiducial on a tripod hanging by its cable – not really a stable environment – and every now and then I would get this weird, spurious data jump, and it took me several days to figure out that Jeri, who sits about ten feet from me, would stomp or tap her foot on the table or the floor, and the vibration went through the floor to the tripod and wiggled that fiducial ever so slightly.

Jeri: At that point we thought, “Holy smokes, we’re picking up vibrations through the floor, this thing is super accurate,” and that’s when we took a step back. We hadn’t even hooked it up to our projection glasses yet, and we hooked it up to a little thumbscrew thing so we could measure the precision and realized, “Oh my god, this thing is out-performing far better than we expected.” We rushed to put it on a pair of glasses, and tried it on and said, “Oh…we’ve just done it.” Everything we tried before that point was just jumpy and no good. I think when the tracking was done, we knew - this is it. We’re going to be able to do this. And then it was just shrinking and testing, shrinking and testing. And now heading forward beyond Kickstarter, we have to focus more on production. The core electronics are there; there are a few bugs we need to work through, but not serious issues. It’s doing tests like using a piece of flex cable to send the data rate down to the displays… it’s just constant baby steps up until the last minute, and at some point (that’s the hard part of engineering that frustrates me at times) you’ve got to pull the plug and say, “I’m done. It’s done. I see that there’s improvements but this is it.”

Thank you Jeri and Rick for your time; we can't wait to see how the castAR project evolves!