Hello, everyone!

We hope you’re all doing well and keeping healthy. In case you haven’t been following our Facebook group for the weekly bite-sized updates, we decided to compile them all here as well. Plus, we added some exclusive bits that you get to find out about here first.

We’re super excited to share new footage, previews, and extra details about our AI training process, as well as some of the social features of the Bird Buddy app. We also made it a point to let you all know where we stand with the firmware testing (and what that looked like in the past month): spoiler alert, it’s good news! Without further ado, let’s get into it:

Firmware testing

The good news!

First up, we wanted to bring you up to speed on what we’ve been up to in terms of firmware testing which is an integral part of how Bird Buddy will function and continue to improve going forward.

We’re very happy to report that we were able to eliminate the last critical bug related to the OTA functionality and its updates, which means that our extended testing of firmware came to a point where we feel confident enough to start the manufacturing process on schedule!

The factory in Shenzhen saw a complete shutdown due to Covid-19 last week, which delayed the start of production by a few days; however, as they are now open and running, we are glad to say that we’re starting with smaller production runs to iron out all processes (this will help us further test the manufacturing processes themselves, along with logistics, and any automatisation necessary for these stages).

We will gradually scale production during the coming weeks to full capacity as planned, which keeps us on schedule for fulfillment starting this May. We’ll be able to get a video of the manufacturing process from the factory in the coming days too, so definitely stay tuned for that!

Why firmware testing was and continues to be a necessary process:

IoT or  “The Internet of Things” is a collective network of connected devices and the technology that facilitates communication between them, which is how Bird Buddy operates as well. There are a lot of background processes that simply must work consistently and continuously all the time in order for your device to function as it should. We want to provide constant improvements in regards to functionalities or features. In order to do that, we need to ensure that our firmware is fully operational in two key capabilities that are absolutely critical:

• OTA or “over the air” updates which enables the device to accept updates remotely.
• Provisioning which is essentially an onboarding phase for your Bird Buddy where you pair your mobile device to Bird Buddy and provide it with the credentials for your Wi-Fi network.

Without these two features, we would be unable to ensure optimal performance remotely (that is, being able to update your devices even once the Bird Buddies are delivered), which would cause your device to be perpetually stuck in the same state you receive it in. These two firmware functionalities will live on with the product and will enable us to add more features to Bird Buddy as time goes on with confidence that they will work as intended.

More than 40 tests are currently ongoing just for the OTA and Provisioning functionalities. To ensure this testing is up to par and done as efficiently as possible, we even built our own in-house testing rig; we designed and 3D printed a holder for the PCBAs (printed circuit boards), which you can see in the photos below:

What does the testing process look like?

The Raspberry Pi (also called “RPi”, which can also be seen on the rack) runs a test framework (called pyTest) which is coded for all the test cases. An example of such a case would be “your WiFi goes out while you’re taking a photo”, or “your battery dies while you’re in the photo flow”.

All the tests defined in pyTest are run parallely on 3 separate PCBAs; each test would normally take between 20 and 40 minutes, so this allows us to mitigate the necessary time, as well as enables us to run multiple tests simultaneously. This in turn makes it possible to run as many tests as we can, to eliminate as many bugs as possible.

The RPi also has functions such as “relay”, which enables it to turn the power on and off, and has its own Wi-Fi module which simulates a user’s Wi-Fi router. Both of these functions are necessary for the most prominent test cases. Another useful feature of the RPi is that it collects logs of all the tests, builds reports from them, and produces alerts for our team to address as soon as possible.

The firmware tests will of course continue in-house indefinitely and will continue to be a part of our quality assurance for new features and functionalities of Bird Buddy going forward.

AI training

Artificial intelligence and machine learning are amazing tools, but the result is only as good as the input given when training it. We wanted to spotlight our team’s behind-the-scenes efforts that make the magic possible.

All the images that we get from our AI training network and our wonderful volunteers go through four stages of processing to ensure the best results:

1. First, we determine whether the image is interesting or not. What’s interesting is subjective, of course, but for now, the AI model considers the image interesting if the visiting bird is in the frame and in focus, and at least one of its eyes and the beak are visible.
2. Secondly, the photo goes through a bird detector model (which we also trained) that marks the bird in the photo and crops it out.
3. In the next stage, we determine the species in the picture and further sort the cropped photos. With the help of advanced tagging and cropping tools, we classify them as either interesting, not interesting, or invalid (phantom detections). This is where our amazing AI trainers step in and do all the legwork. Led by Rebeka (our Head of Special Projects) and Dejan (our in-house ornithologist from the University of Ljubljana), the team tirelessly processes pictures from our AI training network.
4. At least 2 thousand images per species are then introduced into the neural network. They are the material based on which the model learns to distinguish between species. This is how Bird Buddy knows to take a snapshot when a bird appears in its field of view: the potential of this technology is only limited by the data we chose to feed it, so it will only get more accurate and complex as time goes on.

So far the team has processed millions (yes, millions!) of images, fine-tuning the AI to recognize bird species that regularly visit bird feeders. Once Bird Buddies are in use by all of you, our wonderful community members, you’ll be helping us train the AI even further; as mentioned above, the user-generated content will provide an invaluable amount of additional data that we can use to improve both the span and the accuracy of the model (as well as explore options for other recognition processes).

Our AI in action

We've made a considerable effort to improve on our AI by focusing on its ability to specifically recognize birds that visit feeders, and have thus managed to greatly improve its accuracy.

Other than a cute screen letting you know that a bird is being recognized, you won't really see what is happening behind the scenes. That is why we have prepared a visualization of how our AI interprets each individual frame, to give you some insight into how our bird recognition software works in real time. You will not see this process in the Bird Buddy App; however, we thought it might still be interesting to those of you that are curious!

Social aspects of Bird Buddy

Taking adorable pictures of feathery visitors is a super fun activity on its own, but what can make it all even better is sharing the fun with your family, friends, fellow community members, and followers on social media.

Sharing your Bird Buddy photos

Since your Bird Buddy will likely be set up in your backyard, we wanted to make sure that your privacy is taken into account; you’ll always be in control of what you feel comfortable sharing from within the app. You’ll be able to select whether you want to share your photos or not after every photo-taking session, and nothing will be published automatically or without your knowledge.

We will be adding more community features gradually. But here is a glimpse of what we have planned for the very first version of the app:

• After you’ve taken your photos and completed the photo selection flow, you can choose whether you want to share your final photos from the session with the community or not. Since we want to increase safety, you will only be able to share photos with recognised birds in them. Deciding which sets of photos you want to share is as easy as toggling the option on and off.

• The photos will be published in the community tab on each bird species’ profile. You will also be able to share each photo (yours or from the community) via a classic share button (to Facebook, Twitter, Instagram, email etc.). If you share someone else’s photo, it will automatically be credited.
• You’ll be able to get notifications on your personal feed every time someone from the community likes your photo(s).
• You will also be able to report any photo you come across in the community tabs that potentially violates our community guidelines.

Sharing access to your feeder

This is one of our very favorite features - each feeder owner will be able to share direct access to their Bird Buddy with 5 family members, friends, or loved ones. There’ll be no need to share your account details (like your username or password to Netflix for example) to do so either: all you’ll need to do is send them one of your five available invite codes! 😊

You can manage all users you granted access to in the app, for each feeder that you own. If you open your feeder settings, you’ll be able to create an invite code and share it right from the app.

The code recipient will need to create a profile in the app and insert the code. No worries, though, you’ll still have the final say: before they get access to the camera in your backyard you will need to confirm their request. This is just an extra precaution to prevent any unexpected or unwanted visitors from accessing your camera.

After you grant them access, they will be able to connect to the live feed from your Bird Buddy and receive any postcards with the pictures of your visitors.

The team behind the Bird Buddy app would love your input regarding these features as well. If you have a moment, taking this survey they prepared would help them out a bunch. Thank you in advance to everyone who decides to participate!

Hopefully you’re as excited as we are about the awesome news. As always, thank you so much for your invaluable feedback and ongoing support! We’re looking forward to hearing your thoughts in the comments section below.

‍

With warm wishes,
The Bird Buddy team