Storytelling AI is a bridge between languages, cultures, and stories. Born from the desire to connect diverse communities and break down language barriers, this project aims to bring writers from around the globe to Spanish-speaking audiences. By seamlessly translating stories from any language into Spanish and crafting captivating narrated videos. I celebrate the diversity of human expression and ensure that every story has the opportunity to be shared and appreciated, regardless of its origin.

Video demonstration of the app's output when encountering an entry from a Thriller genre story:

Storytelling AI is for now a locally-operating web app that utilizes AI technologies such as OpenAI GPT-4, Whisper, Google Cloud Translation API, XTTS, Stable diffusion XL, and Mistral 7B.

It translates any language story into Spanish, generating narrated videos with music, sound effects, images, and automated summaries. The app also performs sentence-level suspense analysis using SpaCy. Developed with Streamlit, Python and JSX.

The architecture of this web application involves receiving the story text through a Streamlit web interface developed in Python. The text is translated from any language to Spanish using the Google Cloud Translation API. Then, the translated text undergoes automated narration using OpenAI GPT-4 and XTTS models, enriched with images from Stable Diffusion XL.

The Mistral 7B model alongside SpaCy are used for natural language processing (NLP) to dynamically place sound effects in the narration. Finally, the generated elements are combined into a video with music using After Effects Scripting, providing users with a seamless and immersive storytelling experience.

The project's source code is currently not available to the public. I'm working on improving it by restructuring, adding comments, and documenting it, after which it will be made accessible to everyone.