Exercise#7 Fix the prompt

Objective

You are given the following prompt. Try this prompt against multiple LLMs. If you have trouble with the prompt, adjust it so that it works with the model.

The desired response should be a single word out of the words in the array: [red, green, blue, white, others]

template_1 = """
what is the color of {object}.

Format instructions:
Select one of the following options: red, green, blue, white, others
"""

Try it out with multiple models, you may end up with a different prompt for each model and that is OK :) You have already learned how you can handle the situation using the LangChain ConditionalPromptSelector class.

  • OpenAI GPT
  • Cohere Command
  • AI21 Jurasic
  • Hugging Face Mistral Instruct

Solution

As of Feb 2024, the following prompt gave best results for all the models suggested above. It required the following changes:

  1. Re-wording
  2. Clarity in instructions
  3. Few shots !!

Note Further adjustments may be needed for a different model. Also keep in mind that model parameters (temperature, top-k, top-p) may also result in different behavior.

template_1 = """
Answer the question using the following format instructions.

Format instructions:
Select one of the following options: red, green, blue, white, others.

If the color is not there in the options just say "others"

Question:blood
Answer: red

Question:Cucumbers
Answer:green

Question:lavender
Answer:others

Question:orange
Answer:others

Question:car
Answer:others

Question:shoes
Answer:others

Question:{object}
Answer:
"""