ImageGenerator

How can we implement AI image generation? Here are a few options:

Paid:

• OpenAI API (DALL·E 3 model) or other APIs

Free:

• gpt4free (client.images.generate)
• Own generator

Speaking of free options, for better reliability, speed, and quality, we can build our own generator. Such a generator is demonstrated in the bot's demo.

It’s built using browser automation with Playwright and one of the free AI generators based on the DALL·E 3 model.

Next, I’ll guide you on how to write such a generator in the ImageGenerator class.

Playwright + Bing Image Creator

1. Create a Microsoft Bing account.

2. Go to https://www.bing.com/images/create.

3. Copy cookies using the EditThisCookie extension.

4. Save the cookies to a cookies.json file in the data folder of our bot.

5. In the constructor of the ImageGenerator class, create an attribute that holds the path to the cookies file.

6. Write a method that contains the driver object. Example of driver setup:

playwright = sync_playwright().start()
browser = playwright.chromium.launch()
context = browser.new_context()

Options like headless mode can be set in the browser object, and you can disable the AutomationControlled flag.

You can also set the User Agent and change the viewport in the context object.

Return the context from the method.

7. Create an attribute containing the driver object, and assign it the result of the method.

8. Create an attribute for the browser page: self.page = None. This is necessary for accessing the page object in other methods.

...

Writing the login method:

1. Open a new page:

self.page = self.driver.new_page()

2. Check if the cookies file exists. If it does, read its contents (for this, write a separate method) and load them into the browser:

self.driver.add_cookies(cookies)

Writing the execute method:

1. Navigate to the page https://www.bing.com/images/create.

The page URL can be set as a class attribute self.URL:

self.page.goto(self.URL)

2. Find the selector for the text input form.
3. Enter the text (received as an argument) into the form.
4. Find the selector for the "Create" button.
5. Click the button (or simulate pressing the Enter key).
6. Find the selector for the block containing the generated images.
7. Wait for this block to appear:

self.page.wait_for_selector(selector)

8. Find the selector for the images inside the block.
9. Extract the image URLs and download them into the folder for the specific scene.

Data for passing to the execute method can be prepared like this:

1. Loop through the CSV files with prompts from the prompts folder and read their contents.
2. Return a list of lists. Each list contains prompts for a separate script, and each prompt is an element of the list (a string or a dictionary value).
3. Loop through each list (representing a script).
4. Loop through each element (prompt) and pass it to the execute method.

Hint 1: When the cookies expire, obtain new ones and resave them in the cookies.json file.

Hint 2: Look for selectors using the browser inspector (Elements tab).

Hint 3: After creating an instance of the ImageGenerator class in the main.py file, first call the login method and then the execute method.

Hint 4: Extract full-size images of 1024x1024, not thumbnails.

Project structure after saving images:

1. In the generated_images folder, folders are created for each script.
2. Inside these folders, there are subfolders for each scene.
3. In the folder for each scene, the images generated based on the prompt for that scene are saved.

All that's left is to write two simple classes for generating voiceovers and downloading footage, after which we will move on to the final and most interesting class - VideoGenerator.

VoiceGenerator

For generating voiceovers, we will use the ElevenLabs API and its Python library:

https://github.com/elevenlabs/elevenlabs-python

Here’s how this class can be implemented:

Class constructor:

1. Set the API key, voice, and model parameters in the constructor. Set the default value for the voice parameter to, for example, 'Brian', and for the model parameter to 'eleven_multilingual_v2'.

2. Create an instance attribute that contains the client object. To do this, import the ElevenLabs class, create its instance, and pass the API key as an argument:

self.client = ElevenLabs(api_key=api_key)

3. Create an instance attribute self.voice

4. Create an instance attribute self.model

...

Method execute:

1. Accept the text for the voiceover.

2. Send a request to generate the voiceover by calling the generate method on the self.client object. Pass the text, voice, and model as arguments (image). Store the result in a variable. You can write a separate method for this.

3. Form the path to the save folder. This is the folder corresponding to the scene of the script where the generated images are stored.

4. Save the file to this folder using the save function (from elevenlabs import save).

Data for passing to the execute method can be prepared like this:

1. Read the data from the file(s) where the divided script scenes are saved (script_scenes.csv).

2. Return a list of lists. Each list is a script. Each scene is an element of the list (a string or a dictionary key).

3. Loop through each list (script).

4. Loop through each element (scene) and pass it to the execute method.

Hint 1: Obtain the API key on the ElevenLabs website and save it in a .env file. Then, when creating a VoiceGenerator object in the main.py file, retrieve the API key from environment variables through the Config class attribute (Config.ELEVENLABS_API_KEY) and pass it to the class constructor.

Hint 2: You can get the available voice names on the page https://elevenlabs.io/app/speech-synthesis/text-to-speech or by using the method voices:

self.client.voices.get_all()

After generation, the voiceover files for each scene of each script should be saved in the same folders where the images are saved.

FootageDownloader

To download footage, we will use the Pexels API.

Developing this class will teach you how to work directly with service APIs.

Working with the Pexels API (or any other API) typically involves the following steps: obtaining an API key, reviewing the documentation, sending HTTP requests, and handling the responses.

Let's go through these steps using the Pexels API to download videos.

1. Obtaining an API Key

Register on the Pexels website and obtain an API key from your account.

https://www.pexels.com/api/

2. Reviewing API Documentation

The documentation provides all available endpoints, request parameters, example responses, and header requirements.

https://www.pexels.com/api/documentation/

3. Sending Requests

Requests to the Pexels API are sent via HTTP, most often using the GET method to retrieve data. For each request, you need to include a header with your API key and additional parameters.

Example request:

• URL (endpoint): https://api.pexels.com/videos/search

• Header: Authorization: <your_key>

• Request parameters:

- query — search term (e.g., "nature").
- per_page — the number of results returned per page.
- page — page number for pagination.

4. Handling the Response

The API response contains JSON data, which must be processed to extract the video file links.

Let's start developing the FootageDownloader class.

We will insert the video footage in the middle of our Shorts.

A 40-second Shorts usually consists of 6-7 scenes. Therefore, we will save the footage file in the folder for the 3rd or 4th scene.

Class constructor:

1. In the class constructor, accept the API key.

2. Create an attribute self.api_key and assign it the value of the passed argument.

Next, create a method for sending a request (make_request):

1. The method takes an endpoint and request parameters.

2. Then, the headers for the request are formed. This is a dictionary containing the key 'Authorization' and the value of the API key.

headers = {'Authorization': self.api_key}

3. A get request is made to the endpoint, with the parameters and headers set.

4. The method returns the JSON response.

An example of such a method is shown in the image

Method execute:

1. Accept an argument containing the search keyword for the video (query), and also set the parameters page, per_page, orientation with default values:

def execute(self, query, page=1, per_page=10, orientation='portrait'):

2. Form the endpoint path:

endpoint = 'https://api.pexels.com/videos/search'

Alternatively, set it as an object attribute: self.endpoint.

3. Create a dictionary with parameters:

params = {'query': query, 'page': page, 'per_page': per_page, 'orientation': orientation}

4. Send a request to the target endpoint by calling the make_request method and passing the endpoint and parameters.

5. Obtain the JSON response from the request.

6. Extract video links from the JSON response.

7. Save several videos in the folder for the 3rd or 4th scene of the script.

Way to obtain key phrases for a scene:

1. Read the file(s) containing the divided scenes of the script (script_scenes.csv).

2. Take the text of the 3rd or 4th scene for each script.

3. Create a writer object and send a request to the AI assistant, asking it to return a list of key phrases from the text. Request the format to be a Python dictionary or list.

4. Extract the elements of the dictionary or list.

5. Loop through the elements, for each element call the execute method and pass the key phrase into it.

Before developing the final class VideoGenerator, the project structure should look as follows.

VideoGenerator

To implement this class, we will need the following tools:

• Pillow – for image processing (scaling and cropping)

• Moviepy – for merging video clips and audio files

• numpy – for creating animations and converting images into arrays

Command for installation:

pip install pillow moviepy numpy