This is jigsaw.
he's a friendly little robot that's really really good at only one thing putting together any jigsaw puzzle,
no matter how complicated,
really really fast.
it's taken us three years to get to this point,
but according to our initial tests,
we have hopes he might be two hundred times faster than the fastest competitive jigsaw puzzler in the world.
so today we're gonna walk through what it took to get ready for the ultimate robot versuss human face off and along the way we may just discover some tricks you might find helpful as a mere human jigsaw puzzler yourself.
but before we unpack how jigsaw does what he does,
I first want to give us humans some well deserved credit because while this seems pretty straightforward of me to be able to pick up and arrange these twelve pieces of a puzzle,
I'm actually doing four very complicated tasks.
the first is just picking up a piece.
have you ever stopped to think just how amazing our hands are?
hiding beneath that skin are twenty seven bones and thirty four muscles which makes them flexible and strong,
but they're also incredibly precise and dexterous thanks to the high concentration of.
Nerves in our fingertips are sensing pressure,
textures,
and temperature.
we'VE also evolved to have opposable thumbs,
which makes it way easier to hold tools and to manipulate and pick things up than if it was just five fingers all side by side.
step two is rotating the piece to the correct orientation,
which again is pretty straightforward when you have all the abilities.
I just mentioned for step three,
we need to move the piece into position and that requires our whole arm.
as I mentioned before,
most vertebrae have the same basic ARM bone configuration from a human to a BAT to a chicken to a turtle to a dolphin.
but when you includelu the hand,
our configuration is the most technically capable ARM of any living thing to have ever existed on this planet.
if you imagine a large cube in front of me,
it's wild that us humans can move this puzzle piece to any position and orientation within that cube.
that's really hard for a robot or pretty much any other species to do for that matter.
for step four,
we have to decide where this piece should go and for us humans it's hard to explain how,
but when we look at this,
it just very quickly feels super.
Obvious this piece should go here.
what's actually happening,
though,
is our eyes communicate visual perception to our brains,
which then subconsciously synthesize a complicated combination of pattern recognition,
spatial reasoning,
visual memory,
and executive function.
and as a result,
in fractions of a second,
the right answer just feels obvious.
and not to brag,
but this is once again where us humans are the undisputed champions.
our complex brains are what make us special.
physically,
we're kind of unremarkable.
in the animal kingdom,
we're not faster than cheetahs or're stronger than bears.
we can't swim as well as dolphins or fly like an eagle.
it's our brains that make up two percent of our body weight,
yet they consume twenty percent of our energy every day.
and that ratio is higher than any other living thing ever.
and it's the reason we're the ultimate Apex predators because it allows for the huge survival advantages that come from tool use,
planning,
problem solving,
language,
and large scale cooperation with other humans.
it's also what has possibly,
up until this moment in history,
made us the best.
At jigsaw puzzles.
so if we wanted to make a robot that was as good or better than us at puzzles,
our daunting challenge was to take the two hundred million years of evolution that enabled those four steps and figure out how to translate it into things a robot can do.
for number one,
to pick up a piece in lieu of an opposable thumb in twenty seven hand bones,
we used a tiny specialized suction cup that's often used to manipulate small objects on assembly lines.
this solenoid here can cut off and then connect to this vacuum pump,
which means we could turn on the suction exactly when we want to pick it up and then turn it off exactly we want to let it go.
for number two we attached the suction cupped gbber of jigsaw to this very fine tuned donughnut motor that's precise to an incredible point zero zero five degrees.
that means if you attached an infinitely sharp knife to the end of the motor it could slice your birthday cake into over sixty five zero pieces.
for number three to move a piece around we modified our avid CNC router we're constantly using around here for our large builds by upgrading the motors to clear path industrial servo motors.
these are the same motors we used on the dominator.
Our autonomous domino robot,
once we did this we saw it could accurately place the puzzle piece down to point zero,
zero,
five inches.
that's one tenth the width of a human hair,
which means as you can see here,
jigsaw can take the lead out of a mechanical pencil and move all around the table and then come back and put the lead right back in.
so now that jigsaw can pick up,
rotate and move any piece with terrifying precision,
the only thing he lacked was step four,
knowing exactly where to place the pieces.
and as you might guess,
this was by far the hardest one to solve because all the subconscious work performed by neural pathways in our brain that handle pattern recognition and spatial reasoning that make finding the right piece feel so obvious to us is a really,
really hard problem to solve using just computer logic and code.
to make matters worse,
just as we were really struggling to come up with a good solution to this problem,
the most devastating thing that could happen as a youtuber actually happened.
my friend shane from the Youtube channel stuff made here released a video about,
you guessed it,
a machine that solves jigsaw puzzles and what you should know about shane is he's probably
The most technically capable engineer I know if there are like fifteen categories that make you a good engineer,
my knowledge goes pretty deep in maybe four and I'm a generalist and the rest.
but then I have people like Ian and others on my team who helped fill in the gaps.
she however,
is somehow an expert in all fifteen,
which is why you should subscribe to his channel.
when I told him he beat me to the finish line,
he told me he personally wasn't satisfied with where he landed the project and encouraged me to keep trying.
so I persevered and that's when our fortunes improved cause while filming a video in Rwanda covering the work done at zip line where they deliver lifes savinging blood using autonomous drones,
I got to spend some time with Ryan,
their Co founder,
who was responsible for all their complicated software algorithms.
and after hearing all of our challenges,
by the time the blade landed on the way back home,
he'd coded up a solution that ended up being the backbone that unlocked everything for us.
basically,
instead of trying to compete with our brain's very complicated pattern matching neural network,
we took a much more simple approach by completely ignoring what was printed on the puzzle and just looked at the edges.
so while painting a white makes it more impressive and way harder to solve.
For us humans it would make no difference to jigsaw.
so to do this edge analysis we first needed a set of eyes,
but not those eyes.
those are just decorative because it's no secret I love a good pair of googly eyes and these were our most advanced set of googly eyes ever.
because the googly is controlled by two servo motors,
jigsaws realized we're just a cell phone camera.
the idea was to run a serpentine pattern over all the pieces and take a picture of each one and then isolate them against the background,
then convert the edges of every piece into four splines.
so if we wanted to find a match,
say for this edge we would just need to take that spline and then find the puzzle piece that had a corresponding edge spline that matched up perfectly because matching pieces share identical edges.
now even for a small simple puzzle like this,
that's over two hundred edge comparisons that need to be made.
so the first step is just to look at the overall length of each spline and disregard any edges that were significantly longer or shorter and that typically reduced the solution space by about fifty percent.
then for the remaining candidates we would overlay the splines and then quantify the mismatch.
By calculating the area between the two edge splines.
so two pieces that were obviously not a fit would have a lot of overlapping area whereas two pieces that were a perfect match,
the overlap area would essentially be zero.
from there we looked at all the possible spline matchups and ranked pieces,
potential matches with each other from least overlap area to most then jigsaw would start with one of the four corner pieces and map out the potential solution space which you could picture visually like this.
so the correct solution is the only one that finds a good fit for all twelve pieces of the puzzle but as you can see there's lots of forks and dead ends and that's because sometimes it was really obvious what the matching piece was but sometimes there were three or four close contenders because the edges are so similar and our pictures weren't always perfect.
so jigsaw would always picked what seemed like the best match for a set of two pieces but at some point the puzzle would only be halfway done and there would be no more good candidates to match the edges to continue solving the puzzle.
and that's a dead giveaway.
he chose the wrong piece somewhere previously up the chain,
so he would work his way back.
To the closest fork and then choose the next best option.
and he would keep doing this over and over until he found a path that finally connected all twelve pieces with essentially zero spline mismatches at every connection.
and then at that point because he took pictures of all the pieces,
he knows exactly where each of them is starting from over here and he could simply assemble the puzzle using the precision he demonstrated back in steps one through three.
so I was working on a twelve piece puzzle,
but now for the real challenge,
could we scale that up to a one thousand piece all white puzzle?
since our ultimate goal was to dominate the world's fastest human puzzle solver,
we figured maxing out the number of pieces would give the advantage to the non overwhelmable robot.
so after a couple more long weeks filled with plenty of late nights working through all the challenges that come from scaling up a simple prototype a hundred times over,
we landed here.
it's three am.
after three years,
we're at this moment about to fully solve it for the first time.
what's our confidence level?
I'd say one hundred percent.
wow.
the code always works on the first try.
and so with that very heavy dose of sarcasm,
jigsaw got to work.
Starting first with taking all the pictures and that took about one and a half hours,
then after we had the pictures and we did the prep work on them,
I mentioned earlier it was time for jigsaw to actually try and solve the one thousand piece puzzle and astoundingly while running on just a simple laptop,
the actual time to figure out the correct placement of all one thousand pieces was less than a minute and I love how you can actually see him working his way around the puzzle from the top corner,
solving and then going back each time he hits a dead end to try and find a different fork in the solution space until he finds the only combination that makes them all fit perfectly.
so at this point all that was left for jigsaw to do was get to work placing the pieces since he now knew where every single one needed to go and things were looking really good and just as we made the mistake of having some kind of glimmer of hope,
a single piece didn't quite snap into place and then a bunch more followed right behind and upon troubleshooting what we discovered was there were a few very small sources of air that tended to compound on each other the more the puzzle was assembled,
so for example,
there's.
Always a little slop between any two puzzle pieces which adds up over a lot of pieces or another source of error is the whole puzzle itself shifting slightly when certain pieces were laid down and this led us to realize jigsaw was actually still missing one final important feature us impressive humans possess because if you think about it,
when we assemble a puzzle,
we first just approximately placed the piece where it needs to go and then we rely on those highly sensitive nerves in our fingertips to form a feedback loop with the brain to make really tiny adjustments until we feel the piece fall into place and then a double tap is the customary move to make it official,
but since jigsaw doesn't have nerves in his gripper,
we approximated that with the z height encoder on a spring loaded linear slider.
and that encoder is so accurate.
if you slice the human hair lengthwise into fifty pieces,
jigsaw could feel that tiny hair slice if it was resting on top of the table.
so now if jigsaw went to place a piece and he could feel it hadn't quite snapped into place because it was resting too high,
he would employ a wiggle routine where it would just barely translate the piece into various directions.
Jlie got the feedback from his finger that the piece was fully set down,
at which point he would give it the customary final tap like any puzzle solver worth his salt.
and so with jigsaw's new hardware and software upgrades he was ready for one more final attempt.
and I should mention by the way,
from a combined hardware software perspective,
this is by far the most challenging build ever on my channel and that includes the auto bullseye dart board and the automatic domino robot dominator.
so if you really want to appreciate all the juicy details,
including all of the source code,
you can find a link to the full write up in the video description you boys ready to try this again?
officially for the forty eighth time better work this time what's the confidence levels that one hundred percent have you learned?
nothing Ian hit the dang button so jigsaw got to work and after taking pictures of all the pieces he once again actually solved the whole puzzle in a mere fifty five seconds which meant it was time to put them all together and thanks to the new wiggle routine you got this jigsaw first twenty percent was knocked out in less than an hour okay,
I guess we can sit back down and that meant while jigsaw was doing all the hard work,
I just get to kick back.
Can relax in a state of zen gazing at this polar gantry sand gardenard that you can build yourself because unlike the crunch lab's build box which is made for kids,
we just launched hack pack which is kind of similar,
it's just more advanced and created specifically for teenagers and adults.
with hack packck you get a series of really fun programmable robots delivered right to your door where we build it together and learn step by step the kinds of engineering skills that go into making the builds on my channel.
they're all super fun and even useful and they'LL work right out of the box.
no programming required,
but since my goal is to get you to always level up your skills with some simple code tweaks we walk you through,
you can level up the functionality of your robot or get totally creative with some hacks of your own.
so if you want to enhance or even just take the first step of unlocking the really fun and rewarding hobby of making stuff,
just go to crush livesbs dot com or use the link in the video description where we're giving away one free box as an early subscriber special and back over to jigsaw.
things were looking very promising for the worldw.
Record attempt of an off the shelf puzzle completely solved and assembled by a robot with no human intervention.
Some pieces would go right in who know wow some required a little bit of wiggle oh actually yeah and some required a lot of wiggle but in the end let's gos jigsaw always came through yes not feeling more confident in jigsaw's abilities it was time for him to meet the greatest jigsaw puzzler the human race had to offer what are you done?
who fun fact also happens to be a national sudoku champion?
Tammy mcleod and the human benchmark the cing to my child I first wanted her to face off against the best jigsaw puzzler I personally know my dear friend kristen bell AKA Anna or veronica Mars or Eleanor or a bunch more kristen ever since I told you I was making a puzzle robot you'VE been talking so much trash.
little bit of trash.
yeah,
the puzzle robot isn't here today so instead I have a friend I want you to challenge.
this is tammy.
she holds the Guinness world record with the fastest puzzle ever solved by a human being.
hi kristen,
we kick things.
With a simple thirty piece puzzle and to be honest,
three,
two,
one,
go!
christianristen started off pretty strong.
so where's the elephant Tammy?
you don't talk while you're doing it.
I mean I could trash talk but do it!
you're gonna lose Tammy and then you hit me back.
wow,
that was one minute and five seconds for a thirty piece puzzle.
I'm sweating so much and that was a pretty decisive win but I wanted to see one more face off only this time with a five hundred piece puzzle and a huge advantage in kristen's favor.
I think I'm gonna help you on this one.
Okay and with kristen's renewed confidence the clock started and out of the gate Tammy was so confident in his two v one matchup she's just doing flipping oh my god that's crazy organized.
she saw no problem in helping our cause where's that little guy?
his tail right she's right mark,
can I ask you a sincere question?
is this the robot?
admittedly I did have an ulterior motive for this matchup and that was espionage.
Tammy,
I noticed you're using the box what?
hey Tammy,
get real.
you have to look at the picture,
it' just distract me.
I look at the textures.
I have good color acuity,
I have very bad shaped memory so if you say I had to go against an all white puzzle that would maybe be my Achilles heel and this was a very useful bit of information but she didn't stop there because she also told me the four tips she tells amateur jigsaw puzzlers to help them solve puzzles faster.
first off,
dump out all the pieces and turn them over so you can see them all at the same time.
for number two,
most people start with the edge pieces and that's not a bad move,
but it's not always the best,
for example,
if there's a puzzle that.
Something like a gold frame around the outside,
you should set all the edge pieces aside and then leave them till the end where there's a lot more information from the interior pieces to help with the border.
for tip three,
look for groupings that catch your eye and organize them into piles.
so for example,
you could organize by colors,
textures or patterns,
but either way you're trying to reduce the search space into smaller chunks because scanning and pattern matching are different parts of the brain and this way you don't have to do them at the same time which makes it easier.
and finally,
if you're down to a bunch of pieces that all look the same like a blue sky,
sort by shape based on how many ins and outs they have and orient them all the same way.
this will leave you with six total piles and will make it much easier to find the final matches and I need to look no further than our current matchup.
oh no,
I'm looking away for one minute.
don't look at Tammy's Chris the things have gotten dire to understand there must be some validity to these four steps not seeing your hands move or your eyes get I'm not trying to I'm not thirty four minutes and two seconds,
five hundred pieces.
congratulations.
Congratulations!
your next puzzle match will not be against mere humans and indeed,
after setting things up at crunch labs,
we were ready to go.
Tammy,
I'd like to officially introduce you to jigsaw.
this is my upgrade from kristen and jigsaw was in no mood for conversation as he got straight to work.
all right then,
as Tammy entered the matchup of her life with the weight of all humanity resting on her capable shoulders,
what's your strategy here?
the only shape I can make use of is the edge.
okay,
how are we doing jigsaw?
not really sure what I'm supposed to do now,
but that's what I remembered.
I had a specialized skill set of maya.
I might need to take measurements on your neck later.
I'm making you something that's why you don't have any pieces placed yet on your teeth wow,
Tammy,
you're familiar with the saying measure once,
cut twice?
no,
that's not it and as if on cue,
my teammate wrapped up his measuring there's a corner piece and began cutting yeah,
did youe to make him so noisy?
three pieces,
four pieces?
yeahmy,
ignore me.
sing.
Five pieces it ain't over till it's over you're still winning Tammy,
don't let it get to your head and while I was tempted to count all one thousand nine pieces,
I decided to take up woodworking instead are you nervous Tammy?
even if I lost today,
my family would still love me jigsaw if he lost today,
would you still love him?
no,
my love is very conditional having properly reminded jigsaw what was at stake,
jigsaw has now officially taken the lead jigsaw continued to make steady progress while Tammy firsts it lost edge oh,
you need one edge fe face the limits in here in biology I really want to actually help find this for you that way I could say you needed my help without the precise search capabilities I found it I'm done with the edge of a smartphone to laptop nervous system click love that sound anyway I could get some water jigsaw do you need a water break?
you notice we are missing a piece in the middle that's on purpose.
I was the ideas guy and jigsaw was the muscle jigsaw.
that's your next piece right there buddy on the team.
the only thing is he can be a bit of a control freak.
So I opted to help Tammy instead.
Tammy,
do you need this piece?
do you need this?
and so while jigsaw continued to widen the GAP,
I made myself useful by woodworking,
solitary,
finding waldo Tammy right here later and just generally referring to the role of my youth as the annoying youngest sibling.
I love doing puzzles and after all that,
Tammy had made a lot of progress.
Tammy,
I have good news and bad news.
the bad news there's only eight pieces left,
but not enough.
the good news is I made you a scarf.
in fact,
if you assume it takes one second per side to try all the remaining combination of pieces she had left working twelve hours a day,
including weekends,
it would take her one and a half months to finish,
which made this a good time to throw in the towel.
unless you're about to finish,
I think I'm good.
oh,
this is a moment,
one thousand pieces in four hours.
okay,
Tammy,
you represented us humans well,
but alas.
It's time we welcome our benevolent robot overlords.
it's an honor to lose to him that was beautiful I guess say good work jigsaw edd's so while jigsaw took his victory lap,
I put the final touches on the trophy I'd carved her.
you are the best the human race has to offer and you did a great job just,
you know,
not that great if you're a teenager or adult and you'VE always wanted to make him build cool stuff but just haven't figured out that first step,
this is it.
it's called the crunch lab's hack pack and it's basically a series of really fun programmable robots that get delivered right to your door where we'VE build it together and learned step by step the kinds of engineering skills that go into making the bills on my Youtube channel and they all work with no programming required.
but since my goal is to take you from wherever you're at and level you up,
you can easily hack the micro controltrolller brains of any of these robots in a bunch of ways to completely level up the functionality.
there's also a community where you can share your builds or post questions,
as well as an AI chatbot name.
Robot that will check your code for you and help you implement your most creative ideas.
on top of all that,
each box has a chance to contain a platinum diploma.
if your box has it,
congratulations because college is now free for you or a loved one you want to transfer it to.
plus you get to come out here and brainstorm one of your own ideas with me and my team for a day.
so if you want to enhance or even just take the first step of unlocking the really fun and rewarding hobby of making stuff,
just go to crunch labs com or use the link in the video description or're to say thank you,
we're giving away that free box as an subscriber special for watching.