Project: Creative writing workbench (v3)
References
Objective
You are working with a UI developer to build the Creative Writer Workbench application. You are responsible for the backend of the application.
- Review the UI requirements
Part-1
- Design the JSON schema classes
- Setup output parser (JSONOutputParser)
Part-2
- Create the 4 chains
- Create SequentialChain using the given sequence diagram
- Create a wrapper object for UI. It should represent the final JSON. This will get returned to the UI.
- Optional step : Create an API that can be invoked from the UI.
High level architecture
On this project you are a backend developer, the front end will be developed by another engineer. You are responsible for providing the REST-API only.
- Application connects with a web app
- Application sends a request to generate the content to the web app
- Web app invokes the REST/JSON API
- REST/JSON API invokes the LLM and returns the response back
Note
Team has intentionally decided to keep the version 1 simple. In a later iteration, the front end will make a direct call to the REST/API.
Workbench Application : UI Requirements
-
User of the application will provide the writeup instructions in a text box and click on the generate button to carry out the generation process by way of interactions with an LLM. The process is built with Tree of Thought approach that you have built in the past exercises. The output from the LLM at every stage of the processing gets presented to the user.
-
Here are the 3 plans that were generated. The best voted plan is marked with a green checkmark.
-
Three passages gets generated and then the best voted passage is presented to the user. Notice the greeen checkmark. The user has the ability to look through all the plans and the passages.
-
User can click on the “Plan Reasons” button to see the reasons why LLM picked the plan.
-
User can click on the “Passage Reasons” button to see the reasons why LLM picked the passage.
JSON output from the backend
- Refer to the JSON structure below
- This is the JSON schema you need to use for output from your API
- Checkout how the JSON will be drive the UI components
Sequential Chain Requirements
- Study the sequence diagram below
- Each box represents an LLMChain
- Set the output_key from each LLMChain as depicted in the flow
- Output from the SequentialChain will consist of all of the outputs
Hints:
- For each chain create a Python class representing the output
- Carry out incremental development
- Create a LLMChain
- Unit test it by passing as input the output of previous chain
- Continue with the process till you have created all 4 LLChain
Issues?
Every LLM is different. If you face issues with responses, here are a couple of things you may try:
- Adjust the prompt
- Include a few shots / examples of how the output should look like
- Adjust the model parameters (temperature, top-k, top-p)
Solutions
Part-1 JSON classes & Output parser
template_1, JSON classes, & output parser
template_1 = """
you are an expert in writing creative marketing material.
Generate a list of writing plans for executing the task with the following instructions.
The size of the plans list will be three. Keep the number of steps for each plan between 4 and 10.
{task_instructions}
Format Instructructions:
{format_instructions}
"""
# 1. Define the JSON classes for plans & list
class PassagePlans(BaseModel):
plan_number: int = Field(description = "this is the plan number")
plan_steps: list[str] = Field(description = "these are the plan steps")
class ListPassagePlansOutput(BaseModel):
plans: list[PassagePlans] = Field(description = "list of PassagePlans")
# Create the parser
output_parser_generate_plan = JsonOutputParser(pydantic_object=ListPassagePlansOutput)
template_2, JSON classes, & output parser
template_2 = """
you are an expert in writing creative marketing material.
three experts created separate writing plans for carrying out the following task.
your job is to review the three plans, identify the best plan out of the three and provide three reasons for selecting the plan.
you must strictly follow the format instructions to generate your output.
Task Instructions:
{task_instructions}
Plans:
{proposed_plans}
Format Instructions:
{format_instructions}
"""
# Create the class that represents output of step 2 i.e., best voted plan
class VoteBestPlan(BaseModel):
plan_number: int = Field(description = "this is the plan number for the plan voted as best")
plan_steps: list[str] = Field(description = "these are the plan steps")
reasons: list[str] = Field(description = "provides 3 reasons why the plan was voted the best")
# Create the parser
best_plan_parser = JsonOutputParser(pydantic_object=VoteBestPlan)
template_3, JSON classes, & output parser
template_3 = """
You are a writer for marketing material.
Create three passages using the following task instructions and the provided plan.
You must strictly follow the format instructions for the generated passages.
Task Instructions:
{task_instructions}
Plan:
{best_voted_plan_details}
Format Instructions:
{format_instructions}
"""
# Output for generating 3 passages from the best plan
class ThreePassgesWithBestPlan(BaseModel):
passages: list[str] = Field(description = "list containing the generated passages")
# Create the parser
generate_passages_parser = JsonOutputParser(pydantic_object=ThreePassgesWithBestPlan)
template_4, JSON classes, & output parser
template_4 = """
you are an expert in writing creative marketing material.
three experts were given a writing assignment with task instructions.
your task is to review the 3 passages and pick the best passage out of the three.
provide three reasons why you selected the passage.
you must strictly follow the format instructions for the generated passages.
Task Instructions:
{task_instructions}
Passages:
{generated_passages_with_best_plan}
Your output will be in the following format.
Format Instructions:
{format_instructions}
"""
# Define schema for output from last step
class BestPassage(BaseModel):
passage_number: int = Field(description = "this is the passage number for the best passage out of the three passages")
reasons: list[str] = Field(description = "three reasons in a list on why you selected the passage")
# Create the parser
best_passage_parser = JsonOutputParser(pydantic_object=BestPassage)
Part-2 Create the SequentialChain
Please refer to the notebook.
Next 2 lessons in the course provide the walkthrough of the 2 part solution.
Google colab
- Make sure to follow instructions for setting up packages
