Generative AI technologies are essential to producing the best results.
In This blog post will guide you through the key areas to focus on when training your team to work with generative AI.
Before we get started. A key point to keep in mind is that it's important to understand what AI can and cannot do. AI generates outputs based on patterns it has learned during its training. It does not possess emotions or personal experiences, nor does it understand content like humans do. Knowing this will help set realistic expectations and use the tool effectively.
Prompting is the primary method of interacting with generative AI. Effective prompts require specificity, clarity and detail.
Prompting is an iterative process. Once you review the initial output, you will be able to see what was missing in your original prompt. System instructions, like asking the AI to think step-by-step, can be used to generate more thoughtful responses. Experiment, be creative, and learn from each interaction.
Generally, a successful prompt contains:
Role: Telling the AI how to behave (i.e. you are a social media expert speaking to mid-level cybersecurity experts)
Command: Telling the AI what you want (i.e. a social post for LinkedIn to generate engagement no more than 50 words in length)
Additional instruction: Giving the AI additional instructions and context to get exactly the type of response you want (i.e. no more than 3 emojis, use an energetic tone of voice, summarize information from this specific blog, etc)
There are a ton of great prompting resources out there, here is one to get you started.
Control the AI Output
The length and detail of an AI output can be controlled using temperature and max token parameters.
Temperature controls the randomness of the AI's responses. With a range from 0 to 2, a lower temperature will make the output more deterministic and focused, while a higher temperature will make the model's output more diverse and creative adding more randomness to the generated text.
High temperature is better suited for creative output (i.e. brainstorming) while low temperature is better for structured tasks (i.e. email).
While lower temperature makes the output more consistent and focused, it can also make it more repetitive. By the same token, a higher temperature makes the output more varied but can also lead to unexpected and less coherent results.
Max tokens sets a limit on the output length. For instance, if you set max tokens to 50, the model will stop generating output after 50 tokens, regardless of whether it has completed the idea or not. This controls the wordiness of the output, but if set too low, it may cut off the output, leading to incomplete or nonsensical responses.
A token usually corresponds to a word or a character. The exact definition varies based on the language and the model's training.
In the screenshot below, the first prompt has low temperature and tokens while the second has higher temperature and tokens.
AI isn't perfect. It makes mistakes. It hallucinates. It is often too wordy. AI output should always be checked. It is imperative that whoever is doing the QA knows that they are QAing content generated by generative AI so they are focused on the right elements.
It begins with domain knowledge. Without this, your QAer won't ask the right questions when testing the content for accuracy.
Some important things to keep in mind when you are QAing AI output:
- Understand the AI’s Limitations: AI generates outputs based on the patterns it has learned during its training. It's important to critically evaluate its output and not assume it's correct just because it sounds plausible
Check for Accuracy: The AI might generate incorrect or misleading information, especially for niche topics. Whenever possible, fact-check the output against reliable sources
Evaluate Coherence and Relevance: Are the AI's responses coherent and answer the prompt? You may be getting grammatically correct texts explaining nonsensical or off-topic information
Look for Repetitive or Generic Responses: Generative AI models sometimes generate repetitive and/or overly generic responses. Keep your eyes peeled!
Check for Inappropriate Content: Make sure the AI isn't generating offensive, harmful, or inappropriate content. Implement mechanisms to filter or block such content
Collect Feedback: Share the content with users to get their feedback and insights to the quality of the AI's output
Build a Clear Internal Procedure: QAing AI is different. If the reviewer doesn't know they are reading AI generated content then they won't be looking for the right things
Remember, AI will make mistakes. But with the right level of human-AI cooperation you can increase efficiency and productivity.
Integrating generative AI can be a game changer, but only if those using it understand how it works. As with any tool, the key to mastery is practice, experimentation, and continuous learning.
by Yoni Grysman on June 06, 2023
Yoni is our Director of AI marketing solutions and senior marketing strategist. He is certified by the AI Marketing Institute and as a HubSpot trainer. Yoni helps companies adopt generative AI tools in their tech stack and works with AI generated content to produce the ultimate assets in record time. Yoni runs marketing strategy for clients from various industries, including automotive tech, cybersecurity, finance and more. Yoni’s not-so-secret marketing secret? Everything in marketing comes down to goals and audience. If you don’t know who you’re talking to and what you want to achieve, you’re shooting in the dark.