Dear Reader,

Creative writing is experiencing a significant transformation thanks to advancements in Artificial Intelligence. Writers now have the opportunity to use powerful language models directly on their personal devices via tools like LM Studio, dramatically enhancing storytelling, character generation, dialogue creation, and immersive worldbuilding, all without working in someone else's cloud.

If LM Studio sounds familiar, you're right. I wrote about it a few issues ago with regard to anonymising your data before sending it off to the cloud. LM Studio allows writers to run large language models (LLMs) locally, offering privacy, improved performance, and greater control over the writing process. By using quantized versions (like GGUF/GGML), these models can efficiently operate on standard personal hardware, providing real-time feedback and endless creative inspiration.

In other words: it's free, it's local, it's private, and it can be incredibly versatile. In this issue, we'll have a look at using it for creative writing.

Let's get to it.

Choosing an LLM for Creative Writing

When selecting a model for storytelling or role-playing, consider:

Training and Fine-tuning – Models specially fine-tuned on dialogues, stories, or role-play data often produce more natural character interactions and richer world details. For example, some models are tuned on fantasy literature or chat roleplay transcripts.

Model Size vs. Quality – Larger models (e.g. 30B–70B parameters) generally have more fluent diction and coherence (good for complex worldbuilding), whereas mid-sized models (~13B) can be a sweet spot for dialogue and character-centric tales. Smaller 7B models can still perform surprisingly well with the right fine-tuning, and they run faster on limited hardware.

Quantization for Local Use – To run models locally, you’ll likely use 4-bit or 8-bit quantized versions (GGUF/GGML or similar). A 13B model in 4-bit can often fit on a 16GB GPU or even CPU RAM, while a 30B+ model might need 4-bit and high RAM or dual GPUs. We recommend using Q4_K or Q5 quantization for a good balance of speed and output quality in creative tasks. Many models have pre-quantized files available, especially via the LM Studio community and Hugging Face.

Community Feedback – The AI community often discovers which models shine for creative applications. Models that are mentioned frequently in contexts like writing fanfiction, generating RPG scenarios, or AI Dungeon-like storytelling are worth considering due to their proven strengths in those areas.

With the increasing variety of open-source LLMs, writers and hobbyists have a wealth of tools to assist in creative writing. Whether you need a massive 70B model to craft intricate epic sagas, or a lightweight 7B model to run a character dialogue on a phone, there’s likely an open model that fits the bill.

Before deploying any model in your project, it’s wise to experiment – each model has its own “style” and quirks. You might even combine them (e.g. use a smaller model for brainstorming and a larger one for polishing prose). All these models can be run locally via LM Studio with the right format, empowering you to create without relying on cloud APIs. And because they are open-source or community-driven, you can fine-tune or modify them further on your own creative data if needed.

The landscape of open-source LLMs for creative writing is rich and evolving. Models like LLaMA-2 and Falcon provide a high-quality foundation, while specialised models like Pygmalion and MythoMax inject genre-specific prowess (from casual roleplay to mythic fantasy).

Armed with these models on your local machine, you can have a formidable AI writing toolkit to help bring any story universe to life – one dialogue and one description at a time.

Top Open-Source LLMs for Creative Writing ​ Open-source large language models (LLMs) have emerged as powerful tools for creative writing, offering a range of capabilities from generating vivid descriptions to maintaining coherent narratives. ​ Not sure where to start? Check out this guide. ​ Read more

Fun ChatGPT Prompt for Friday Ever wondered what you'd look like as an action figure? Now you can find out! ​ Here's a simple prompt to try in ChatGPT: ​ "Create an image of an action figure in packaging labelled '[Your Name]'. Use the attached photo as reference for the face. [He/She/They] is [your height] tall, dressed in [describe your outfit], and holding a [an item like a coffee mug or phone]. The cardboard section should be [choose a colour]. Include an 'Accessories' section with items like: [list your favourite accessories]. ​ Make the design visually appealing and reflective of your personality or professional background. (Do not include the character image on packaging.)" ​ Try Jackson

I think that getting to know how local AI works is going to be an important skill in the long run. As much as I like using and consulting on the mainstream tools available, it's never a good idea to put all your eggs in one basket, as it were!

If you've had a go at installing and working with LM Studio, I'd love to hear what you're doing with it! Let me know.

Until next time,

Jim

​​​​

Made with ❤️ in Valencia by Jim Christian. For feedback, please reach out to hello@jimchristian.net.

​