A Developer’s Guide to Project IDX and Gemini API

A year has passed since the IDX team revealed their aim for Project IDX: transforming full-stack, multiplatform app development by consolidating essential tools and services within the browser. Many things have evolved since then.

In this blog, I’ll walk you through how to effectively use the Gemini API within Google’s Project IDX, showcasing the power and flexibility of both platforms. This comprehensive, step-by-step guide will help streamline the workflows and boost efficiency, making it an ideal resource for new developers looking to enhance their skills and harness the latest technology. As part of the tutorial, I’ll create a sample app to demonstrate practical integration and implementation.

Source: Project IDX

Before we dive into app creation, let’s take a moment to explore Project IDX and its features.

What is Project IDX?

Project IDX is a platform introduced by Google (announced at Google I/O 2023) to streamline and enhance the development of apps, particularly for those building on the web and mobile platforms. It is designed to be a powerful, cloud-based development environment that integrates various tools and services to simplify the coding and deployment processes.

Key Features of Project IDX:

• Cloud-Based Development Environment: It provides a fully cloud-hosted workspace (like VS Code in the cloud), enabling developers to access their projects from anywhere, on any device.
• Integration with Google Cloud Services: It is deeply integrated with Google Cloud Services, offering developers a robust platform for deploying applications and leveraging cloud functionalities.
• Support for Multiple Language and Frameworks: Flexibility is at the heart of Project IDX. It provides a range of templates to help you quickly launch various projects.🪄

• Web Integration and Analysis: Project IDX not only simplifies cloud-based development but also enhances web integration through tools like Lighthouse. This feature helps developers optimize their applications for performance and, accessibility.
• Emulators and Device Simulation: It offers robust emulator integration and device simulation features, enabling developers to test their applications seamlessly. I think it is one of the best features.

Sample Gemini Flutter App

• GitHub Integration: Seamless integration with GitHub allows developers to manage their code repositories effortlessly. Teams can collaborate on projects, track changes, and deploy updates directly from IDX.

• Customizable Environments & Collaboration Tools: Project IDX empowers developers with customizable environments and advanced collaboration tools, making it easier to tailor workflows and enhance teamwork throughout development.

Let’s Get Started: Project IDX AND GEMINI API

It is quite easy. Go to the Project IDX website and log in using your Google account. Before doing that, obtain a Get API key | Google AI Studio.

There are three different ways to use Gemini with Project IDX.

Way 1: By Adding the Gemini API to a new project

Click on Get Started Now Button

Now, Use a Template to get started on something new. You can utilize templates based on your preferences or skills.

I’m going to use the Node Express template.
The next step is to name our application. I like going to the gym. As a result, today we’ll create a fitness-related app. Let me name it Workout Wise.
(I’ve used Gemini to generate the app name. Isn’t it cool?!)
(P.S: This app will reduce the expense of hiring a personal trainer! Additionally, you don’t need to look up the activities’ names on Google, all the time.😉)

Project Structure

We are going to utilize the Google AI JavaScript SDK. If you’d like to try something new, check out the documentation here: https://www.npmjs.com/package/@google/generative-ai.
Now, we will execute the listed command in the Project IDX terminal.
(To open a new terminal use Ctrl + Shift + ` ).

npm install @google/generative-ai

Or we can add the required dependencies in the package.json file

"dependencies": {
    "@google/generative-ai": "^0.19.0",
    "dotenv": "^16.4.5",
    "express": "^4.21.0",
    "multer": "^1.4.5-lts.1",
    "pdfkit": "^0.15.0"
  }

Then, we can use the npm install command to install all packages at once.
We will use these dependencies for the project just like in the code above.
We’re going to save private keys in a Dot Env file.
(Always keep these details private, and take care to protect your API key)
It’s time to write the code. We’re going to use the Gemini 1.5 Flash model for this project. Depending on your needs, you can utilize any Gemini model. (You can create an app by following the official documentation)

const genAI = new GoogleGenerativeAI(process.env.GEMINI_API_KEY);    
const model = genAI.getGenerativeModel({ model: "gemini-1.5-flash" });
    const result = await model.generateContent([
      "Analyze this image and provide Detailed Information, How to do this workout, Benefits, Care Instructions, and Some Interesting Facts. Please provide the response in plain text without using any markdown format.",
      {
        inlineData: {
          mimeType: req.file.mimetype,
          data: imageData,
        },
      },
    ]);

I have the source code up on GitHub. Feel free to give it a try.
(P.S: Don’t forget to rate this repository, if you found it helpful.)

Once we finish writing the code, we can run it with the following command.

npm start

This will open our application in a web view.

We can open this in a new window by clicking the arrow key ↗️ that is in the top-right corner of the Project IDX window.

Demo of Workout Wise App

Output

Here’s the demo for the functional app. Feel free to check it.

App Demo Available on YouTube

Way 2: By Adding the Gemini API to an existing project

Let’s follow these steps:
Step 1: Open your existing project.
Step 2: Click on the IDX icon from the navigation pane to open the IDX panel.

Step 3: Expand the Google Integrations section and Click on Add Gemini API.

Step 4: Follow the instructions to authenticate your account and add a Gemini API Key to your app.

Once everything is configured successfully it will show like this.

Now you can use Gemini API inside your project. You can also use the chat assistant in the sidebar.

Way 3: By Using Gemini API pre-loaded

It’s quite easy!

GEMINI API TEMPLATE

• Click on Gemini API from the IDX template dashboard.
• Choose your preferred language and framework. The Gemini API template is available in JavaScript (using Vite) and Python (for notebooks or web apps with Flask).
• (Optional) If you want to use the LangChain framework, select the “Use LangChain” option.
• Once your workspace is ready, open the main.js or main.py file and replace the API_KEY variable with your Gemini API key from Google AI Studio.
• Test the app to see the Gemini API working, and make any changes you want to your new app.

Bonus resources for you!!🤩

• IDX Community
• Project IDX Blog
• Project IDX Roadmap
• Welcome to Project IDX | Google for Developers
• Tips & tricks for using IDX: Building with Gemini in IDX
• Building an App with Gemini in Firebase and Project IDX
• The AI-Assisted developer workflow: Build smarter with IDX and Chrome DevTools

If you have tried Project IDX previously or if you would like to test it out, please let me know what you think.

If you enjoyed reading this blog, a round of praise (claps 👏) would be greatly appreciated! Furthermore, remember to click the “follow” button for more future stuff like this. I appreciate it! 💖