Bugfix/yarn lock (#6)

* Create llm and embeddings implementation, include tests (#1) * Initiate repo, start creating llm solution: generate and stream created * Exclude authentication outside of the file * Allow generating text on deployed models, make changes according authentication exclusion * Extract types outside of llms, small change * start working on embeddings. create simple tests * Add some features, rework llm * Add embeddings * Create tests for llms and embeddings * Small fixes * Adjust to sdk new package * Rename variables * Add README.md file * Rename variables --------- Co-authored-by: FIlip Żmijewski <[email protected]> * Add dependencies * fix(langchain): Fix local file store traversal issue (langchain-ai#6736) * fix(ci): Fix cf exports test (langchain-ai#6737) * fix(ci): Fix cf-workers export test (langchain-ai#6738) * fix(ci): Fix esbuild tests (langchain-ai#6739) * fix(ci): Fix ci (langchain-ai#6741) * chore(langchain): Release 0.2.19 * fix(langchain): Prevent exception when using plan & execute agent with no tools (langchain-ai#6744) Co-authored-by: Brace Sproul <[email protected]> * fix(openai): Avoid thrown error on o1 stream calls (langchain-ai#6747) * chore(openai): Release 0.2.11 (langchain-ai#6748) * docs: Update link (langchain-ai#6757) * refactor: Move langchain/core to a peer dep (langchain-ai#6698) Co-authored-by: Brace Sproul <[email protected]> Co-authored-by: Erick Friis <[email protected]> * chore(core): Release 0.3.0 (langchain-ai#6760) * chore(deps): Update deps for 0.3.0 (langchain-ai#6761) * chore(textsplitters): Release 0.1.0 (langchain-ai#6762) * chore(cloudflare): Bump dep (langchain-ai#6763) * chore(openai): Release 0.3.0 (langchain-ai#6764) * chore(anthropic): Release 0.3.0 (langchain-ai#6765) * chore(mistralai): Release 0.1.0 (langchain-ai#6766) * chore(langchain): Release 0.3.0 * chore(build): Update lock (langchain-ai#6770) * chore(cohere): Release 0.3.0 (langchain-ai#6771) * chore(weaviate): Release 0.1.0 (langchain-ai#6773) * chore(aws): Release 0.1.0 (langchain-ai#6775) * chore(azure-cosmosdb): Release 0.2.0 (langchain-ai#6776) * chore(azure-dynamic-sessions): Release 0.2.0 (langchain-ai#6777) * chore(release): Release Baidu, Exa, MixedBread, Nomic (langchain-ai#6778) * fix(ci): Fix CI issue where local monorepo node_modules were copied into export tests (langchain-ai#6774) * chore(community): Release 0.3.0 (langchain-ai#6779) * chore(cloudflare): Release 0.1.0 (langchain-ai#6780) * chore(groq): Release 0.1.1 (langchain-ai#6781) * chore(mongodb): Release 0.1.0 (langchain-ai#6782) * chore(ollama): Release 0.1.0 (langchain-ai#6783) * chore(pinecone): Release 0.1.0 (langchain-ai#6784) * chore(qdrant): Release 0.1.0 (langchain-ai#6785) * chore(redis): Release 0.1.0 (langchain-ai#6786) * chore(google): Release 0.1.0 (langchain-ai#6787) * chore(google-genai): Release 0.1.0 (langchain-ai#6788) * fix(core, langchain): Update langsmith dep to rc (langchain-ai#6797) * chore(core): Release 0.3.1 (langchain-ai#6800) * Release 0.3.1 * fix(langchain): Remove circular peer dep in langchain on @langchain/community (langchain-ai#6803) * chore(langchain): Release 0.3.2 (langchain-ai#6805) * docs: new algolia index (langchain-ai#6810) * docs: Adds 0.3 versioning docs (langchain-ai#6793) * docs: NotFound page suggested links (langchain-ai#6809) Co-authored-by: jacoblee93 <[email protected]> * docs: Fix Firecrawl v0 version in the docs (langchain-ai#6768) * docs: Fix concepts docs hash param (langchain-ai#6816) * fix(google): Fixed - TypeError: Cannot read properties of undefined (reading 'generationInfo') (langchain-ai#6756) * fix(ci): Conditionally deplpy api refs to prod (langchain-ai#6819) * fix(ci): Add vercel ignore file (langchain-ai#6820) * feat(google-common): Export ChatConnection class (langchain-ai#6823) * chore(google-common): Release 0.1.1 (langchain-ai#6824) * fix(core,langchain,community): Update LangSmith deps, lighten export tests (langchain-ai#6825) * deps: update @anthropic-ai/sdk to 0.27.3 from 0.25.2 (langchain-ai#6796) Co-authored-by: jacoblee93 <[email protected]> * chore(anthropic): Release 0.3.1 (langchain-ai#6826) * docs: Adds warnings to cypher chain about credentials (langchain-ai#6827) * feat(community): Remove required param from LanceDB integration (langchain-ai#6706) Co-authored-by: jacoblee93 <[email protected]> * fix(ci): Fix lint (langchain-ai#6828) * chore(community): Release 0.3.1 (langchain-ai#6831) * fix(ci): Update API ref deployment action & vercelignore (langchain-ai#6835) * fix(core): Fix nested stream events behavior (langchain-ai#6836) * chore(core): Release 0.3.2 (langchain-ai#6839) * fix(ci): Add script for aliasing domains in gh deployments (langchain-ai#6838) * chore(deps): bump the pip group across 1 directory with 2 updates (langchain-ai#6840) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * fix(groq): Feature/groq response format improvements (langchain-ai#6754) Co-authored-by: jacoblee93 <[email protected]> * fix(core): Loosen return type of withConfig, fix tracing for nested tool calls (langchain-ai#6847) * chore(core): Release 0.3.3 (langchain-ai#6848) * docs(tutorial): fix typo (langchain-ai#6849) * fix(mistral): Fix call options type (langchain-ai#6857) * chore(mistralai): Release 0.1.1 (langchain-ai#6858) * fix: typo fix in agents doc (langchain-ai#6859) * feat(community): Update ChromeLLM with latest experimental API (langchain-ai#6861) * chore(community): Release 0.3.2 (langchain-ai#6862) * fix(community): bump `@upstash/ratelimit` version (langchain-ai#6832) Co-authored-by: bracesproul <[email protected]> * fix(docs): Drop announcement bar (langchain-ai#6863) * groq[patch]: set api key attr (langchain-ai#6864) Co-authored-by: jacoblee93 <[email protected]> * chore(groq): Release 0.1.2 (langchain-ai#6865) * chore(openai,anthropic): Expose OpenAI and Anthropic payload formatters (langchain-ai#6872) * chore(openai): Release 0.3.1 (langchain-ai#6874) * chore(anthropic): Release 0.3.2 (langchain-ai#6875) * feat(anthropic,openai): Add payload formatting utils to Anthropic and OpenAI (langchain-ai#6876) * chore(openai): Release 0.3.2 (langchain-ai#6877) * chore(anthropic): Release 0.3.3 (langchain-ai#6878) * community[patch]: support AWS_... env vars (langchain-ai#6866) * chore(community): Release 0.3.3 (langchain-ai#6880) * Fix yarn.lock file --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: FIlip Żmijewski <[email protected]> Co-authored-by: Jacob Lee <[email protected]> Co-authored-by: oleg <[email protected]> Co-authored-by: Brace Sproul <[email protected]> Co-authored-by: Erick Friis <[email protected]> Co-authored-by: Nicolas <[email protected]> Co-authored-by: Prakash Pokharel <[email protected]> Co-authored-by: Lab Rat <[email protected]> Co-authored-by: Ayush Chaurasia <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Herman Hrand <[email protected]> Co-authored-by: jjangga0214 <[email protected]> Co-authored-by: Stephen M Abbott <[email protected]> Co-authored-by: Cahid Arda Öz <[email protected]> Co-authored-by: Bagatur <[email protected]>
FilipZmijewski · Sep 27, 2024 · 8950ce2 · 8950ce2
1 parent ca6a5dc
commit 8950ce2
Show file tree

Hide file tree

Showing 28 changed files with 2,692 additions and 5 deletions.
diff --git a/libs/langchain-ibm/.eslintrc.cjs b/libs/langchain-ibm/.eslintrc.cjs
@@ -0,0 +1,74 @@
+module.exports = {
+  extends: [
+    "airbnb-base",
+    "eslint:recommended",
+    "prettier",
+    "plugin:@typescript-eslint/recommended",
+  ],
+  parserOptions: {
+    ecmaVersion: 12,
+    parser: "@typescript-eslint/parser",
+    project: "./tsconfig.json",
+    sourceType: "module",
+  },
+  plugins: ["@typescript-eslint", "no-instanceof"],
+  ignorePatterns: [
+    ".eslintrc.cjs",
+    "scripts",
+    "node_modules",
+    "dist",
+    "dist-cjs",
+    "*.js",
+    "*.cjs",
+    "*.d.ts",
+  ],
+  rules: {
+    "no-process-env": 2,
+    "no-instanceof/no-instanceof": 2,
+    "@typescript-eslint/explicit-module-boundary-types": 0,
+    "@typescript-eslint/no-empty-function": 0,
+    "@typescript-eslint/no-shadow": 0,
+    "@typescript-eslint/no-empty-interface": 0,
+    "@typescript-eslint/no-use-before-define": ["error", "nofunc"],
+    "@typescript-eslint/no-unused-vars": ["warn", { args: "none" }],
+    "@typescript-eslint/no-floating-promises": "error",
+    "@typescript-eslint/no-misused-promises": "error",
+    camelcase: 0,
+    "class-methods-use-this": 0,
+    "import/extensions": [2, "ignorePackages"],
+    "import/no-extraneous-dependencies": [
+      "error",
+      { devDependencies: ["**/*.test.ts"] },
+    ],
+    "import/no-unresolved": 0,
+    "import/prefer-default-export": 0,
+    "keyword-spacing": "error",
+    "max-classes-per-file": 0,
+    "max-len": 0,
+    "no-await-in-loop": 0,
+    "no-bitwise": 0,
+    "no-console": 0,
+    "no-restricted-syntax": 0,
+    "no-shadow": 0,
+    "no-continue": 0,
+    "no-void": 0,
+    "no-underscore-dangle": 0,
+    "no-use-before-define": 0,
+    "no-useless-constructor": 0,
+    "no-return-await": 0,
+    "consistent-return": 0,
+    "no-else-return": 0,
+    "func-names": 0,
+    "no-lonely-if": 0,
+    "prefer-rest-params": 0,
+    "new-cap": ["error", { properties: false, capIsNew: false }],
+  },
+  overrides: [
+    {
+      files: ["**/*.test.ts"],
+      rules: {
+        "@typescript-eslint/no-unused-vars": "off",
+      },
+    },
+  ],
+};
diff --git a/libs/langchain-ibm/.gitignore b/libs/langchain-ibm/.gitignore
@@ -0,0 +1,7 @@
+index.cjs
+index.js
+index.d.ts
+index.d.cts
+node_modules
+dist
+.yarn
diff --git a/libs/langchain-ibm/.prettierrc b/libs/langchain-ibm/.prettierrc
@@ -0,0 +1,19 @@
+{
+  "$schema": "https://json.schemastore.org/prettierrc",
+  "printWidth": 80,
+  "tabWidth": 2,
+  "useTabs": false,
+  "semi": true,
+  "singleQuote": false,
+  "quoteProps": "as-needed",
+  "jsxSingleQuote": false,
+  "trailingComma": "es5",
+  "bracketSpacing": true,
+  "arrowParens": "always",
+  "requirePragma": false,
+  "insertPragma": false,
+  "proseWrap": "preserve",
+  "htmlWhitespaceSensitivity": "css",
+  "vueIndentScriptAndStyle": false,
+  "endOfLine": "lf"
+}
diff --git a/libs/langchain-ibm/.release-it.json b/libs/langchain-ibm/.release-it.json
@@ -0,0 +1,10 @@
+{
+  "github": {
+    "release": true,
+    "autoGenerate": true,
+    "tokenRef": "GITHUB_TOKEN_RELEASE"
+  },
+  "npm": {
+    "versionArgs": ["--workspaces-update=false"]
+  }
+}
diff --git a/libs/langchain-ibm/LICENSE b/libs/langchain-ibm/LICENSE
@@ -0,0 +1,21 @@
+The MIT License
+
+Copyright (c) 2023 LangChain
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.
diff --git a/libs/langchain-ibm/README.md b/libs/langchain-ibm/README.md
@@ -0,0 +1,201 @@
+# @langchain/ibm
+
+This package contains the LangChain.js intergations for Watsonx by IBM thtough their SDK.
+
+## Installation
+
+```bash npm2yarn
+npm install @langchain/ibm
+```
+
+This package, along with the main LangChain package, depends on [`@langchain/core`](https://npmjs.com/package/@langchain/core/).
+If you are using this package with other LangChain packages, you should make sure that all of the packages depend on the same instance of @langchain/core.
+You can do so by adding appropriate field to your project's `package.json` like this:
+
+```json
+{
+  "name": "your-project",
+  "version": "0.0.0",
+  "dependencies": {
+    "@langchain/ibm": "^0.0.1",
+    "langchain": "0.0.207"
+  },
+  "resolutions": {
+    "@langchain/core": "0.1.5"
+  },
+  "overrides": {
+    "@langchain/core": "0.1.5"
+  },
+  "pnpm": {
+    "overrides": {
+      "@langchain/core": "0.1.5"
+    }
+  }
+}
+```
+
+The field you need depends on the package manager you're using, but we recommend adding a field for the common `yarn`, `npm`, and `pnpm` to maximize compatibility.
+
+## Chat Models
+
+This package contains the `WatsonxLLM` class, which is the recommended way to interface with the Watsonx series of models.
+
+To use, install the requirements, and configure your environment depending on what type od authentication you will be using.
+
+
+## IAM authentication
+```bash
+export WATSONX_AI_AUTH_TYPE=iam
+export WATSONX_AI_APIKEY=<YOUR-APIKEY>
+```
+
+## Bearer token authentication
+```bash
+export WATSONX_AI_AUTH_TYPE=bearertoken
+export WATSONX_AI_BEARER_TOKEN=<YOUR-BEARER-TOKEN>
+```
+
+### CP4D authentication
+```bash
+export WATSONX_AI_AUTH_TYPE=cp4d
+export WATSONX_AI_USERNAME=<YOUR_USERNAME>
+export WATSONX_AI_PASSWORD=<YOUR_PASSWORD>
+export WATSONX_AI_URL=<URL>
+```
+
+Once these are places in your enviromental variables and object is initialized authentication will proceed automatically. 
+
+Authentication can also be accomplished by passing these values as parameters to a new instance.
+
+## IAM authentication
+```typescript
+import { WatsonxLLM } from "@langchain/ibm";
+
+    const props = {
+        version: "YYYY-MM-DD",
+        serviceUrl: "<SERVICE_URL>",
+        projectId: "<PROJECT_ID>",
+        watsonxAIAuthType: "iam",
+        watsonxAIApikey:"<YOUR-APIKEY>"
+    };
+    const instance = new WatsonxLLM(props);
+```
+## Bearer token authentication
+
+```typescript
+import { WatsonxLLM } from "@langchain/ibm";
+
+    const props = {
+        version: "YYYY-MM-DD",
+        serviceUrl:  "<SERVICE_URL>",
+        projectId: "<PROJECT_ID>",
+        watsonxAIAuthType: "bearertoken",
+        watsonxAIBearerToken:"<YOUR-BEARERTOKEN>"
+    };
+    const instance = new WatsonxLLM(props);
+```
+### CP4D authentication
+
+```typescript
+import { WatsonxLLM } from "@langchain/ibm";
+
+    const props = {
+        version: "YYYY-MM-DD",
+        serviceUrl: "<SERVICE_URL>",
+        projectId: "<PROJECT_ID>",
+        watsonxAIAuthType: "cp4d",
+        watsonxAIUsername:"<YOUR-USERNAME>",
+        watsonxAIPassword:"<YOUR-PASSWORD>",
+        watsonxAIUrl: "<url>"
+    };
+    const instance = new WatsonxLLM(props);
+```
+
+## Loading the model
+You might need to adjust model parameters for different models or tasks. For more details on the parameters, refer to IBM's documentation.
+
+```typescript
+import { WatsonxLLM } from "@langchain/ibm";
+    const props = { 
+      decoding_method: "sample",
+      max_new_tokens: 100,
+      min_new_tokens: 1,
+      temperature: 0.5,
+      top_k: 50,
+      top_p: 1,
+    }
+  const instance = new WatsonxLLM({
+      version: "YYYY-MM-DD",
+      serviceUrl: process.env.API_URL,
+      projectId: "<PROJECT_ID>",
+      spaceId: "<SPACE_ID>",
+      idOrName: "<DEPLOYMENT_ID>",
+      modelId: "<MODEL_ID>",
+      ...props
+    });
+```
+Note:
+- You must provide spaceId, projectId or idOrName(deployment id) in order to proceed.
+- Depending on the region of your provisioned service instance, use correct serviceUrl.
+- You need to specify the model you want to use for inferencing through model_id.
+
+
+
+## Props overwrittion
+Passed props at initialization will last for the whole life cycle of the object, however you may overwrite them for a single method's call by passing second argument as below
+
+```typescript
+const result = await instance.invoke("Print hello world.", {
+  modelId: "<NEW_MODEL_ID>",
+  parameters:{
+    max_new_tokens: 20,
+  }
+});
+console.log(result);
+```
+
+## Text generation
+
+```typescript
+const result = await instance.invoke("Print hello world.");
+console.log(result);
+
+const results = await instance.generate(["Print hello world.", "Print bye, bye world!"]);
+console.log(result);
+```
+
+## Streaming
+
+```typescript
+const result = await instance.stream("Print hello world.");
+for await(let chunk of result){
+  console.log(chunk);
+}
+```
+
+## Tokenization
+This package has it's custom getNumTokens implementation which returns exact amount of tokens that would be used.
+
+```typescript
+const tokens = await instance.getNumTokens("Print hello world.");
+console.log(tokens)
+```
+
+## Embeddings
+  Following package supports embeddings model, you can proceed with following code snipet.
+
+```typescript
+  import { WatsonxEmbeddings } from "@langchain/ibm";
+
+  const instance = new WatsonxEmbeddings({
+        version: "YYYY-MM-DD",
+        serviceUrl: process.env.API_URL,
+        projectId: "<PROJECT_ID>",
+        spaceId: "<SPACE_ID>",
+        idOrName: "<DEPLOYMENT_ID>",
+        modelId: "<MODEL_ID>"
+      });
+
+  const result = await instance.embedQuery("Hello world!");
+  console.log(result);
+```
diff --git a/libs/langchain-ibm/jest.config.cjs b/libs/langchain-ibm/jest.config.cjs
@@ -0,0 +1,21 @@
+/** @type {import('ts-jest').JestConfigWithTsJest} */
+module.exports = {
+  preset: "ts-jest/presets/default-esm",
+  testEnvironment: "./jest.env.cjs",
+  modulePathIgnorePatterns: ["dist/", "docs/"],
+  moduleNameMapper: {
+    "^(\\.{1,2}/.*)\\.js$": "$1",
+  },
+  transform: {
+    "^.+\\.tsx?$": ["@swc/jest"],
+  },
+  transformIgnorePatterns: [
+    "/node_modules/",
+    "\\.pnp\\.[^\\/]+$",
+    "./scripts/jest-setup-after-env.js",
+  ],
+  setupFiles: ["dotenv/config"],
+  testTimeout: 20_000,
+  passWithNoTests: true,
+  collectCoverageFrom: ["src/**/*.ts"],
+};
diff --git a/libs/langchain-ibm/jest.env.cjs b/libs/langchain-ibm/jest.env.cjs
@@ -0,0 +1,12 @@
+const { TestEnvironment } = require("jest-environment-node");
+
+class AdjustedTestEnvironmentToSupportFloat32Array extends TestEnvironment {
+  constructor(config, context) {
+    // Make `instanceof Float32Array` return true in tests
+    // to avoid https://github.com/xenova/transformers.js/issues/57 and https://github.com/jestjs/jest/issues/2549
+    super(config, context);
+    this.global.Float32Array = Float32Array;
+  }
+}
+
+module.exports = AdjustedTestEnvironmentToSupportFloat32Array;
diff --git a/libs/langchain-ibm/langchain.config.js b/libs/langchain-ibm/langchain.config.js
@@ -0,0 +1,22 @@
+import { resolve, dirname } from "node:path";
+import { fileURLToPath } from "node:url";
+
+/**
+ * @param {string} relativePath
+ * @returns {string}
+ */
+function abs(relativePath) {
+  return resolve(dirname(fileURLToPath(import.meta.url)), relativePath);
+}
+
+export const config = {
+  internals: [/node\:/, /@langchain\/core\//],
+  entrypoints: {
+    index: "index",
+  },
+  requiresOptionalDependency: [],
+  tsConfigPath: resolve("./tsconfig.json"),
+  cjsSource: "./dist-cjs",
+  cjsDestination: "./dist",
+  abs,
+};