Paragon Project: Deep Learning Work Assistant

When it comes to productivity, tinkerers have made use of microprocessors and microcontrollers in a variety of ways.

However, this deep learning work assistant tops the list as one of the most creative projects we have seen in a long time!

This project, developed by Benjamin over at WIZnet Makers, take photos every 5 seconds and then reads them through AI to recognize work patterns.

Benjamin has trained the model to recognize some of the many situations all of us face when working. It identifies when you’re feeling normal or drowsy or if you yawn. More importantly, by scanning your face, it can tell you if you’re distracted and even if you’re using your cellphone.

With all of this information, you’ll be able to determine just how productive you’re really being!

Hardware

For Hardware, Benjamin went with a modified Raspberry Pi Pico clone put out by WIZnet along with an Arducam.

However, you could definitely use a standard Raspberry Pi Pico with another camera instead if need be.

Furthermore, depending on how fast you want this to run, you might consider checking out what a Google Coral could do in making this race at lightning speed!

But with a microcontroller and a camera, you’ll definitely have enough to develop a modified form of this project at the very least.

In my experience, that’s often what makes something like this so exciting: mixing and matching hardware can often spur new results and spin-off projects!

Software

Relying on CircuitPython, Benjamin was kind enough to publish the code for this project on GitHub. So once you install that code on your Pico, things should be up and running.

The code relies on the nano model of Ultralytics YOLOv8, which is a “real-time object detection and image segmentation model”. If you’re interested in the model structure of YOLOv8 more generally, you can find it here.

Meanwhile, if you’re interested in reading up more on deep learning, my colleague Nathan has written a great post all about it here.

With the nano model of YOLOv8, Benjamin runs everything through a Flask application to spit out the real-time information as it tracks someone’s workflow.

Try it out for yourself and see if you can identify your own work patterns!

At least this deep learning work assistant will help answer my lifelong question: How many times do I yawn before my third cup of coffee?

You can find more Paragon Projects here.

What would you do with a work assistant like this?