Prompt Engineering for Generative AI

Future-Proof Inputs for Reliable AI Outputs

By James Phoenix

Category: Science | Reading Duration: 20 min | Rating: 4.2/5 (47 ratings)

About the Book

Prompt Engineering for Generative AI (2024) is a comprehensive guide to working effectively with text and image generating AI. It highlights five key principles that form the backbone of effective prompts for large language models like ChatGPT and diffusion models like Stable Diffusion. It explores the computational mechanisms behind these models, and gives a host of practical tips and ideas for improving your AI workflow.

Who Should Read This?

Content creators using ChatGPT or similar platforms
Business professionals seeking competitive AI advantages
Developers wanting to harness AI tools effectively

What’s in it for me? Become an efficient prompt engineer.

Generative AI models are making leaps and bounds on an almost daily basis. The pace is so relentless it’s hard to keep up. But keep up we must, for one thing is certain: AI is here to stay. As it evolves, industries evolve with it, and it’s certain to infiltrate more and more lines of work in years to come.

That being said, AI isn’t a mind reader – or perhaps, not yet. It won’t always give you what you were expecting. In fact, it will often make things up with the utmost confidence – a phenomenon referred to as hallucination. The quality of what your AI outputs depends on the quality of what you input into it. That’s why you need to be able to create effective prompts – known as prompt engineering. It’s a skill that will likely become a prerequisite for many jobs in the future, much like Microsoft Excel proficiency is today.

Luckily, the authors have been working with generative AI since the early days – way back in the distant past of 2020. They’ve come up with a set of best practices – a number of principles that have proven as useful with the early models as with their newer incarnations. This Blink will lay out those very principles. The authors offer many avenues for prompting in their book, many of which require running code in Python.

In this Blink we’ll focus on the stuff that only requires plain English. But don’t worry – the same principles of prompt engineering apply across the board. So without further ado, let’s open the sprawling AI playbook and get started.

Chapter 1: Three principles of prompt engineering

Let’s start with a simple, naive prompt, such as “Can you give me a list of names for a pair of shoes that will fit any size? ” Input this into a Large Language Model like ChatGPT, and it’ll return you names like OmniFit or Universole. Now, if you think about it, this is already remarkably impressive. But if you intend to use your AI model for more than just entertainment, you’ll want to get a bit more specific.

Cue the five principles of prompt engineering. These are timeless principles the authors have pinned down – and it turns out they apply to any type of intelligence, whether human or artificial. The first of these principles is to always give direction. Put simply, the more detail you put into your input, the higher the chances that your model’s output will match your expectations. A great way to put this into practice is via a technique called prewarming, or internal retrieval. In the case of the shoe names, you would start by prompting your AI model to give a comprehensive list of tips for naming products, based on expert advice.

You would then prompt your model for the product names, based on the advice it just provided you. Giving direction is equally important when it comes to image-generating models. If your prompt is for an image of a business meeting around a round glass table, you will get a more specific output than by simply asking for an image of a business meeting. Be careful, though: get too specific, and your AI can run into conflicting combinations that it can’t resolve. The second of our prompting principles revolves around specifying the format. AI models are able to output data in almost any format, from French to Klingon, from JSON to Python and beyond.

As for image generation models, they can produce stock images, oil paintings, mosaics, Minecraft imitations, and countless more. But if you don’t specify the format you want, your chances of receiving it in your output decrease. And if you’re integrating AI tools in production software, you’ll need to be extra careful to specify the format. After all, small deviations can lead to all sorts of errors. So, to sure up your chances of getting an output appropriate for your goals, make use of the third principle of prompt engineering: provide examples. Prompts that don’t include examples are called zero-shot prompts.

If you couple your prompt with one example, you get a one-shot prompt, and if you include more than one, you get a few-shot prompt. The more examples you provide, the more predictable the output. But remember – there’s always a trade-off between reliability and creativity. Provide too many examples that are lacking in diversity, and your AI won’t be able to surprise you – in a good way or bad.

Chapter 2: Two more prompting principles

Let’s now unpack the fourth of the authors’ prompting principles: evaluate output. For one-off prompts that you won’t revisit again, simple trial and error is usually enough to get you where you need to be. This is known as blind prompting. But if you’re going to use your prompt repeatedly or you’re building an application based on a prompt, you’ll need to go a little bit further.

In many cases, the solution for evaluating output is pretty simple: a thumbs-up/thumbs-down rating system. Take a selection of prompts you want to test and run them each through your AI model multiple times. Then lay out the outputs next to each other in a spreadsheet and give them a thumbs up or a thumbs down. If you want to get more fine-grained, you can also use a numerical rating score instead of thumbs. It’s also possible to automate the spreadsheeting part using the OpenAI Python package, though we won’t be able to cover the exact steps here. You can evaluate images similarly, with a technique called permutation prompting.

This is when you give your AI model multiple prompts, with different directions and formats, and then lay out all the model’s outputs side by side in order to compare them. This helps you focus on what actually works moving forward. Alright – we’ve now reached the fifth and final prompting principles, and this is all about dividing labor. Think about it like this: as you add more elements to your prompt, you end up asking more from your AI model in a single call. At a certain point, elaborate prompts can become too heavy, leading to more hallucinations and anomalies. That’s when what’s known as task decomposition becomes important.

Just like with humans, breaking down complex tasks into more manageable chunks for AI can lead to more effective results. It also allows you to see exactly which chunks are working well and which aren’t. Even including the words “let’s think step by step” in your prompt can actually help a lot. This is an important component of what’s known as chain of thought reasoning. It turns out you can even divide labor between text and image generating models. You already have a list of names for a shoe that can fit any size.

Now prompt your LLM to give you a detailed description of those shoes. You can then use that description as a prompt for your image generating model, creating a suitable image for your product. So, there you have it – the five prompting principles. As a whole, they’ve stood the test of time, at least so far. Yes, what counts as the latest technology today may well become obsolete in the space of a few months. But the authors are confident that these principles will always help you get what you want out of your AI models.

Chapter 3: How LLMs work

It’s a general rule that you can get more out of something if you understand how it works. So, let’s now embark on a daunting task: understanding large language models, or LLMs for short. It all starts with what’s called the token. This is the fundamental unit in natural language processing – or NLP.

Tokens can represent sentences, words, or even sets of characters which form subwords. For a rough idea of size, a text of 100 tokens is about 75 words long. Why are tokens important, you might be asking? Well, to prepare data for NLP tasks, LLMs first need to perform tokenization. One commonly used method is byte-pair encoding, or BPE. This process builds a vocabulary by analyzing frequency.

It starts with single characters and iteratively merges the ones that appear together most often. Eventually, a frequent sequence like the “c,” “a,” and “t” that spell “cat” becomes a single token. This strategy gives the LLM immense flexibility – it stores common words as whole units, but can still process rare or brand-new words by assembling them from these smaller, known components. Once tokenized, these units are converted into lists of numbers called vectors, or word embeddings. These embeddings capture both the structure and meaning of a word. During training, the LLM maps these vectors onto a massive, multi-dimensional grid.

Words with related meanings, like “swimming” and “swam” or “immoral” and “sin,” land physically close to one another. This proximity allows the model to understand the logical relationships between concepts. The engine processing these vectors is the transformer, and its defining feature is a mechanism called self-attention. This allows every word in a sentence to analyze every other word simultaneously, ensuring the model grasps the full context rather than just reading strictly left-to-right. So, that’s how LLMs understand text. But how do they generate their own words?

It all comes down to probabilities. In a nutshell, the model calculates how likely it is that a given token will follow the current sequence of tokens, and picks the token that is most likely. It then combines this new token with all the previous tokens and starts the process again, running the text through the transformer and generating the next token. This loop continues until the response is complete. Alright – that’s how LLMs work. Now, let’s look at some ways to get the most out of them.

Chapter 4: Simple practices in text generation

Before you can prompt your LLM, you need to choose which one to use in the first place. There are many models to choose from, each with their own computational architecture, strengths, and weaknesses. Of course, you have the big powerhouses like Open AI’s ChatGPT, Anthropic’s Claude and Google’s Gemini, but there are also smaller models like Meta’s Llama and even Mistral, a model by French start-up Mistral AI. The surest bet here is to run the same prompt through several of these LLMs and see which output works best for you, all for your specific task.

A word of caution, though: data privacy is a real concern when using any of these models. If the models use your own data for retraining or fine-tuning, be careful and don’t input any sensitive information. Once you’ve chosen your model, there are a number of simple techniques that can help you get the most out of them – especially when used alongside the five prompting principles you learned earlier. One of these techniques is called text style unbundling. This is when you prompt your model to extract key features from a text, such as tone, vocabulary and structure. This can be incredibly useful for generating new content while maintaining brand consistency, or for adapting existing material to new styles or formats while preserving its key message.

One way to unbundle a text style is to define the specific features you’re looking for and prompt your AI model to analyze the text based on those features. But another way to do it is to use meta prompting, which is when you create a prompt which itself generates another prompt. You could put this to use by asking your model to analyze a text, identify core features that will allow you to imitate it, and present these core features as a style writing guide for you to use for future documents. You could then include this writing guide as part of your next text-generating prompts. Another important type of meta prompting involves asking your model to refine an existing prompt in order to achieve a specific goal. You could, for example, ask your model to examine your style writing guide and improve it, for the purpose of achieving a consistent brand identity across texts.

Role prompting also allows you to maintain a consistent style across outputs. This is when you give your AI a specific role or character to play. For example, you could prompt your AI to act like a tech reviewer, or even imitate the speech style of Donald Trump or other well-known figures. Apart from brand consistency, role prompting can encourage your AI to give you different viewpoints on a topic, or infuse a little humor in your output to enhance user engagement. Just make sure to continuously evaluate your AI’s output – it can easily begin to drift away from its assigned role over time.

Chapter 5: How image generation works

They say a picture is worth a thousand words. Well, with AI, you can generate thousands of pictures with just a few words. But how exactly does it work? The most popular AI models that generate images from text are called diffusion models.

As of late 2025, these include OpenAI’s GPT Image, Google’s Nano Banana, and open-source alternatives such as Stable Diffusion. During training, diffusion models take a clear image and slowly ruin it by adding random pixel fuzz – technically called noise – until the image is just a blur. They then learn to reverse this process, cleaning up the fuzz step-by-step to get back to the original picture. This cleanup is guided by a text description. If the restored image doesn't match the description, the model corrects itself. This cycle runs billions of times.

Once trained, the model can take a square of pure random fuzz and refine it into a detailed image based on your instructions. This process teaches the model the visual patterns that define objects and styles. It saves these patterns as lists of numbers, or vectors, just like LLMs do with words. These vectors act as specific locations in a latent space – a huge multi-dimensional map that holds every possible image combination the model can imagine.

When you give the model a text prompt, it converts your words into these vector coordinates. It generates the image that exists at that location on the map, and finally converts that data into pixels on your screen. There are many diffusion models out there, each with their own strengths. Just like with text generating models, the best way to decide between them is to use the same prompt across different models and compare the outputs.

Chapter 6: Image generation techniques

We’ll finish with a couple of tricks of the image-generating trade. One great tool when using diffusion models is reverse engineering prompts. Sometimes you might know what you want your output to look like, but you don’t know quite how to put it into words. Luckily, models like Image GPT and Nano Banana allow you to upload an image and ask it to give you its description.

You can then take this prompt and tweak it to get your desired output. You can also include quality boosters in your prompt. These are simple expressions like “beautiful”, “high resolution” or “trending on ArtStation” that can be included in your prompt to improve the quality of the resulting image. Negative prompting is when you include something that you want your model to avoid. So, for example, prompting “Tom and Jerry, no cartoon” would give you a realistic depiction of the two famous characters. If you want to get into more minutiae with your prompts, certain models allow you to add weights to your terms.

All words in a prompt have a weight of 1 by default, though what comes first in the prompt has a greater effect. You can explicitly change this to get unique results. In models like Midjourney, you do this by adding two semi-colons and a number denoting your desired weight, without any spaces. So for example, you could input this: “painting of Tom and Jerry, in the style of Rembrandt::0. 8, in the style of Pollock::0. 2.

” This would give you a painting which is mostly Rembrandt but with a pinch of Pollock. The number of possible prompts you can create is infinite, and as AI continues to evolve, new options will become available. But with the principles you’ve learned in this Blink, you’ll be ready to make the most of whatever comes next.

Final summary

The main takeaway of this Blink to Prompt Engineering for Generative AI by James Phoenix and Mike Taylor is that you can make the most out of text and image generating AI by following five essential principles of prompt engineering: give direction, specify format, provide examples, evaluate output, and divide labor. You’ve also learned about the underlying mechanisms of large language models, and some simple tactics you can use to make the most out of them. We then moved on to looking at how diffusion models work, and what works best when operating them. Armed with all this knowledge, you can now navigate the ever-evolving world of generative AI with confidence and skill.

Okay, that’s it for this Blink. We hope you enjoyed it. If you can, please take the time to leave us a rating – we always appreciate your feedback. See you in the next Blink.

About the Author

Mike Taylor co-founded Ladder, a 50-person marketing agency that worked with major brands like Unilever and Nestle. He’s also created courses on LinkedIn Learning, Vexpower and Udemy, all of which have been taken by over 400,000 people. James Phoenix specializes in data pipelines and marketing automation, having taught over 60 data science boot camps on the tech education platform General Assembly. He partnered with Mike Taylor on an Udemy course teaching data science.