NingG +

AI 系列：Software Development toolKits

2023-11-18

原文：Software Development toolKits

Software Development toolKits#

LLM SDKs are specific for generative AI. These toolkits help developers integrate LLM capabilities into applications. The LLM SDK typically includes APIs, sample code, and documentation to aid in the development process. By leveraging an LLM SDK, developers can streamline their development processes and ensure compliance with industry standards.

Table 12 Comparison of LLM SDKs#

SDK Use cases Vector stores Embedding model LLM Model Languages Features
LangChain Chatbots, prompt chaining, document related tasks Comprehensive list of data sources available to get connected readily State of art embedding models in the bucket to choose from A-Z availability of LLMs out there in the market Python, Javascript, Typescript Open source & 1.5k+ contributors strong for active project development
LLaMA Index Connecting multiple data sources to LLMs, document query interface using retrieval augmented generation, advanced chatbots, structured analytics Wide options to connect & facility to create a new one Besides the 3 commonly available models we can use a custom embedding model as well Set of restricted availability of LLM models besides customised abstractions suited for your custom data Python, Javascript, Typescript Tailor-made for high customisations if not happy with the current parameters and integrations
LiteLLM Integrating multiple LLMs, evaluating LLMs Not Applicable Currently supports only text-embedding-ada-002 from OpenAI & Azure Expanding the list of LLM providers with the most commonly used ones ready for use Python Lightweight, streaming model response, consistent output response

在当前人工智能领域，为什么需要大型语言模型（LLM）软件开发工具包（SDK）有几个原因。

  1. 合规性协议：使用LLM SDK，开发人员可以通过适当记录、追踪和监控请求来确保其应用程序符合协议。这有助于避免与软件盗版或未经授权使用资源相关的潜在法律问题。
  2. 用户体验提升：LLM SDK可以通过消除样板代码，并抽象化与LLM的低层交互，帮助创建无缝的用户体验。
  3. 增强安全性：通过实施LLM SDK，开发人员可以通过访问控制和用户管理（access control and user management.）等安全功能保护其资源，并防止未经授权的软件使用。
  4. 灵活性：LLM SDK提供了定制和整合不同组件的灵活性，使开发人员能够根据自己的特定需求定制管理系统，并轻松进行调整。
  5. 改善协作：LLM SDK可以通过提供集中式许可证管理平台促进团队成员之间的协作，确保每个人对问题和合规要求都了解并保持一致。

LangChain#

banner

On the LangChain page – it states that LangChain is a framework for developing applications powered by Large Language Models(LLMs). It is available as an python sdk and npm packages suited for development purposes.

Document Loader#

Well the beauty of LangChain is we can take input from various different files to make it usable for a great extent. Point to be noted is they can be of various formats like .pdf, .json, .md, .html, and .csv.

Vector Stores#

After collecting the data they are converted in the form of embeddings for the further use by storing them in any of the vector database. Through this way we can perform vector search and retrieve the data from the embeddings that are very much close to the embed query.

The list of vector stores that LangChain supports can be found here.

Models#

This is the heart of most LLMs, where the core functionality resides. There are broadly 2 different types of models which LangChain integrates with:

Tools#

Tools are interfaces that an agent uses to interact with the world. They connect real world software products with the power of LLMs. This gives more flexibility, the way we use LangChain and improves its capabilities.

Prompt engineering#

Prompt engineering is used to generate prompts for the custom prompt template. The custom prompt template takes in a function name and its corresponding source code, and generates an English language explanation of the function.

To create prompts for prompt engineering, the LangChain team uses a custom prompt template called FunctionExplainerPromptTemplate. This template takes the function name and source code as input variables and formats them into a prompt. The prompt includes the function name, source code, and an empty explanation section. The generated prompt can then be used to guide the language model in generating an explanation for the function.

Overall, prompt engineering is an important aspect of working with language models as it allows us to shape the model’s responses and improve its performance in specific tasks.

More about all the prompts can be found here.

Advanced features#

LangChain provides several advanced features that make it a powerful framework for developing applications powered by language models. Some of the advanced features include:

Overall, LangChain’s advanced features enable developers to build advanced language model applications with ease and flexibility. Some limitations of LangChain are that while it is useful for rapid prototyping of LLM applications, scalability and deploying in production remains a concern - it might not be particularly useful for handling a large number of users simultaneously, and maintaining low latency.

LLaMA Index#

banner

LLaMAIndex是一个用于LLM（大型语言模型）应用程序的数据框架，用于摄取、结构化和访问私有或特定领域的数据。

LLaMAIndex还有一些相关项目，如run-llama/llama-hub and run-llama/llama-lab.

Data connectors#

Data connectors are software components that enable the transfer of data between different systems or applications. They provide a way to extract data from a source system, transform it if necessary, and load it into a target system. Data connectors are commonly used in data integration and ETL (Extract, Transform, Load) processes.

There are various types of data connectors available, depending on the specific systems or applications they connect to. Some common ones include:

Data connectors play a crucial role in enabling data interoperability and ensuring seamless data flow between systems. They simplify the process of data integration and enable organisations to leverage data from various sources for analysis, reporting, and decision-making purposes.

Data indexes#

LLaMAIndex中的数据索引(Data indexes)是数据的中间表示形式，以易于大型语言模型（LLMs）消费且具有良好性能的方式进行结构化。这些索引是从文档构建的，作为检索增强生成（RAG, retrieval-augmented generation）用例的核心基础。在内部，LLaMAIndex中的索引将数据存储在Node对象中，这些对象代表原始文档的分块。这些索引还公开了Retriever接口，支持额外的配置和自动化。LLaMAIndex提供了几种类型的索引，包括向量存储索引、摘要索引、树形索引、关键词表索引、知识图谱索引和SQL索引(Vector Store Index, Summary Index, Tree Index, Keyword Table Index, Knowledge Graph Index, and SQL Index)。每种索引都有其特定的用例和功能。

To get started with data indexes in LLaMAIndex, you can use the from_documents method to create an index from a collection of documents. Here’s an example using the Vector Store Index:

from llama\_index import VectorStoreIndex
index \= VectorStoreIndex.from\_documents(docs)

Overall, data indexes in LLaMAIndex play a crucial role in enabling natural language access to data and facilitating question & answer and chat interactions with the data. They provide a structured and efficient way for LLMs to retrieve relevant context for user queries.

Data engines#

Data engines in LLaMAIndex refer to the query engines and chat engines that allow users to interact with their data. These engines are end-to-end pipelines that enable users to ask questions or have conversations with their data. The broad classification of data engines are:

Query engine#

Chat engine#

Both query engines and chat engines can be used to interact with data in various use cases. The main distinction is that query engines focus on single questions and answers, while chat engines enable more dynamic and interactive conversations. These engines leverage the power of LLMs and the underlying indexes to provide relevant and informative responses to user queries.

Data agent#

数据代理（Data Agents）是LLaMAIndex中由LLMs驱动的知识工作者：

  1. 能够智能地执行各种数据任务，包括“读”和“写”功能。
  2. 它们具有自动搜索和检索不同类型数据的能力，包括非结构化、半结构化和结构化数据。
  3. 此外，它们可以以结构化的方式调用外部服务API，并处理响应，以及将其存储以供以后使用。

数据代理（Data agents）不仅可以从静态数据源读取数据，还可以动态地摄取和修改来自不同工具的数据。它们由两个核心组件组成：推理循环和工具抽象（reasoning loop and tool abstractions）。

数据代理的推理循环取决于所使用的代理类型。LLaMAIndex支持 2 种类型的代理：

  1. OpenAI功能代理（OpenAI Function agent）：构建在OpenAI功能API（OpenAI Function API）之上。
  2. ReAct代理（ReAct agent）：可在任何聊天/文本完成端点上工作。

工具抽象是构建数据代理的重要组成部分。

  1. 这些抽象，定义了代理可以与之交互的API或工具集。
  2. 代理使用推理循环来决定使用哪些工具，以什么顺序调用每个工具以及调用每个工具的参数。

To use data agents in LLaMAIndex, you can follow the usage pattern below:

from llama\_index.agent import OpenAIAgent
from llama\_index.llms import OpenAI

\# Initialise LLM & OpenAI agent
llm \= OpenAI(model\="gpt-3.5-turbo-0613")
agent \= OpenAIAgent.from\_tools(tools, llm\=llm, verbose\=True)

Overall, data agents in LLaMAIndex provide a powerful way to interact with and manipulate data, making them valuable tools for various applications.

Advanced features#

LLaMAIndex提供多种高级功能，迎合了高级用户的需求。其中一些高级功能包括：

  1. 定制和扩展( Customisation and Extension)：LLaMAIndex提供了低级API，允许高级用户定制和扩展框架中的任何模块。这包括数据连接器、索引、检索器、查询引擎和重新排名模块(data connectors, indices, retrievers, query engines, and re-ranking modules)。用户可以根据自己的特定需求定制这些组件，增强LLaMAIndex的功能性。

  2. 数据代理(Data Agents)：LLaMAIndex包含名为数据代理的由LLM驱动的知识Worker。这些代理可以智能地执行各种数据任务，包括自动搜索和检索(search and retrieval)。它们可以读取并修改来自不同工具的数据，使其在数据处理方面非常灵活。数据代理包括推理循环和工具抽象(reasoning loop and tool abstractions)，使其能够与外部服务API进行交互并处理响应。

  3. 应用集成(Application Integrations)：LLaMAIndex可以与生态系统中的其他应用无缝集成。无论是LangChain、Flask还是ChatGPT，LLaMAIndex都可以与各种工具和框架集成，以增强其功能和扩展其能力。

  4. 高级API(High-Level API)：LLaMAIndex提供了一个高级API，让初学者只需几行代码就能快速摄取和查询数据。这种用户友好的界面简化了初学者的流程，同时仍提供强大的功能。

  5. 模块化架构(Modular Architecture)：LLaMAIndex遵循模块化架构，允许用户独立理解和操作框架中的不同组件。这种模块化方法使用户能够定制和组合不同模块，为其特定用例创建量身定制的解决方案。

LLaMAIndex似乎更适合在生产中部署LLM应用。然而，行业如何整合LLaMAIndex到LLM应用中，或者开发定制的LLM数据集成方法，这仍有待观察。

LiteLLM#

banner

这个名字很贴切，LiteLLM 是一个轻量级的软件包，简化了同时从多个 API 获取响应的任务，无需担心导入的问题。它作为一个 Python 软件包提供，并且可以通过 pip 访问。此外，我们可以使用现成的 playground 来测试这个库的工作原理。

Completions#

This is similar to OpenAI create_completion() method that allows you to call various available LLMs in the same format. LiteLLMs gives the flexibility to fine-tune the models but there is a catch, only on a few parameters. There is also batch completion possible which helps us to process multiple prompts simultaneously.

Embeddings & Providers#

There is not much to talk about regarding embeddings but worth mentioning. We have access to OpenAI and Azure OpenAI embedding models which support text-embedding-ada-002.

However there are many supported providers, including HuggingFace, Cohere, OpenAI, Replicate, Anthropic, etc.

Streaming Queries#

By setting the stream=True parameter to boolean True we can view the streaming iterator response in the output. But this is currently supported for models like OpenAI, Azure, Anthropic, and HuggingFace.

The idea behind LiteLLM seems neat - the ability to query multiple LLMs using the same logic. However, it remains to be seen how this will impact the industry and what specific use-cases this solves.

Future And Other SDKs#

LangChain, LLaMA Index, and LiteLLM have exciting future plans to unlock high-value LLM applications. Future initiatives from Langchain include improving the TypeScript package to enable more full-stack and frontend developers to create LLM applications, improved document retrieval, and enabling more observability/experimentation with LLM applications. LlamaIndex is developing an enterprise solution to help remove technical and security barriers for data usage. Apart from the SDKs discussed, there are a variety of newer SDKs for other aspects of integrating LLMs in production. One example is prefecthq/marvin, great for building APIs, data pipelines, and streamlining the AI engineering framework for building natural language interfaces. Another example is homanp/superagent, which is a higher level abstraction and allows for building many AI applications/micro services like chatbots, co-pilots, assistants, etc.

原文地址：https://ningg.top/ai-series-prem-10-sdk-of-llm/
微信公众号 ningg, 联系我

同类文章:

微信搜索: 公众号 ningg, 联系我, 交个朋友.

京ICP备19013747号-1
Top