Google has used its own project Astra to swing against OpenAI's GPT-4o This universal AI agent is designed to be your assistant for daily life tasks and utilizes your phone's camera and voice recognition to give a response 

Google also demonstrated Project Astra using smart glasses 

To be clear, Astra is coming to the phone first and will be called Gemini Live, but may move to other form factors over time But I can say that the demo shown in Google I/O2024 is impressive 

Google says Project Astra can understand and respond to the world like people do, understand the context, take what you see and hear to take action, and also speak naturally without experiencing delays to remember

power Project Astra was built on Google's Gemini model and other task-specific models It can process information faster by continuously processing video and speech input 

During a Project Astra demo, a person held an Android phone and left the live video of the camera open, asking a series of questions Project Astra did not miss a beat

For example, when he pointed the phone's camera at the table and asked what made the sound, Astra found the computer's speakers Then the woman circled the top of the speaker and asked Astra what it was She reacted correctly to the tweeter

From there, Astra was able to provide a creative allusion about a bunch of crayons, identify which parts of the code do when directed at a computer monitor, and correctly identify the King's Cross area of London when the camera is directed out the window

Then things got really interesting A Google employee put on a smart glass and asked what this reminds me of when I looked at the board and stared at the illustration What does this remind you of? Schrödinger's cat, Astra, replied

Project Astra was asked to come up with a band name on the spot while holding a stuffed tiger next to the Golden Retriever Answer: Golden stripes Overall, Project Astra's spatial understanding and video processing is impressive, and we're excited to see where Google is adopting this AI agent It's coming to the Gemini app later this year and we can't wait to test it out