Using Ai, how to save cost when the text is too long?

"One of the trick is to use LangChain cleverly. LangChain can be designed to summarize text of any length. To summarize text with an LLM, there are a few strategies.

If the whole text fits in the context window, then you can simply feed the raw data and get the result. LangChain refers to that strategy as the ""stuff ""chain type. Often, the number of tokens contained in the text is larger than the LLM's maximum capacity. A typical strategy is to break down the data into multiple chunks, summarize each chunk, and summarize the concatenated summaries in a final ""combine"" step. LangChain refers to this strategy as ""map-reduce "".

Another strategy is to begin the summary with the first chunk and refine it little by little with each of the following chunks. LangChain refers to this as ""refine "". For example, here is the prompt template used by LangChain for the refine step:

Your job is to produce a final summary.

We have provided an existing summary up to a certain point: existing_answer

We have the opportunity to refine the existing summary (only if needed) with some more context below.

text

Given the new context, refine the original summary

If the context isn't useful, return the original summary. "

Author

AiUTOMATING PEOPLE, ABN ASIA was founded by people with deep roots in academia, with work experience in the US, Holland, Hungary, Japan, South Korea, Singapore, and Vietnam. ABN Asia is where academia and technology meet opportunity. With our cutting-edge solutions and competent software development services, we're helping businesses level up and take on the global scene. Our commitment: Faster. Better. More reliable. In most cases: Cheaper as well.

Feel free to reach out to us whenever you require IT services, digital consulting, off-the-shelf software solutions, or if you'd like to send us requests for proposals (RFPs). You can contact us at [email protected]. We're ready to assist you with all your technology needs.

© ABN ASIA