AI Native 模块

概览

从 OpenSumi 3.0 开始,支持通过集成 AI Native Module 的方式来定制 AI 能力,包括但不限于

  1. 内置的 AI Chat 助手,并提供 Agent 开放能力与快捷指令能力的注册和扩展
  2. Inline Chat 能力开放,在编辑器内通过丰富的交互与 AI 能力来生成或理解代码
  3. 代码补全能力开放,包含块补全、行内补全等基础能力
  4. 问题诊断能力开放,提供程序运行错误或静态语法问题时的检测能力
  5. 智能终端能力开放
  6. 智能解决冲突能力开放
  7. 智能重命名能力开放
  8. ...

未来将持续开放更多 IDE 功能的 AI 开放能力

如何使用

第一步:引模块

首先在 browser 层和 server 层都引入 AI Native 模块。建议搭配 DesignModule 一起使用!

// browser 层
import { AINativeModule } from '@opensumi/ide-ai-native/lib/browser';
import { DesignModule } from '@opensumi/ide-design/lib/browser';

renderApp({
  modules: [...CommonBrowserModules, DesignModule, AINativeModule]
});

// server 层
import { AINativeModule } from '@opensumi/ide-ai-native/lib/node';

startServer({
  modules: [...CommonNodeModules, AINativeModule]
});

第二步:贡献 Contribution

这一步的目的是为了注册各种 AI 能力

1. 新建一个 contribution 文件,并实现 AINativeCoreContribution 接口

import { AINativeCoreContribution } from '@opensumi/ide-ai-native/lib/browser/types';

@Domain(AINativeCoreContribution)
export class AiNativeContribution implements AINativeCoreContribution {
  // 在这里注册各种 AI 能力
}

2. 将其注入到 DI 的 Providers 列表或自定义 module 的 providers 的配置里

const injector = new Injector();
injector.addProviders(AiNativeContribution);

opts.injector = injector;
const app = new ClientApp(opts);

第三部:注册后端服务

这一步的目的是为了让前端的 AI 交互能请求到后端接口的服务

完整示例代码见 ai.back.service.ts

1. 新建一个后端 service 文件

@Injectable()
export class AiBackService implements IAIBackService<ReqeustResponse, ChatReadableStream> {
  // 在这里可以跟任何的大模型 API 做接口交互

  // 例如 request 可以一次性返回大模型的返回结果
  async request(
    input: string,
    options: IAIBackServiceOption,
    cancelToken?: CancellationToken
  ) {
    // TODO
  }

  // requestStream 需要返回一个 ChatReadableStream,以便前端可以流式读取模型的回答
  async requestStream(
    input: string,
    options: IAIBackServiceOption,
    cancelToken?: CancellationToken
  ) {
    const chatReadableStream = new ChatReadableStream();
    cancelToken?.onCancellationRequested(() => {
      chatReadableStream.end();
    });

    setTimeout(() => {
      chatReadableStream.emitData({ kind: 'content', content: 'Hello' });
      await sleep(10);
      chatReadableStream.emitData({ kind: 'content', content: 'OpenSumi!' });
    }, 1000);

    return chatReadableStream;
  }
}

2. 依赖注入 AIBackSerivceToken 将新建好的 AiBackService 文件通过依赖注入的方式提供给 Provider

import { AIBackSerivceToken } from '@opensumi/ide-core-common';

const injector = new Injector();
injector.addProviders({
  token: AIBackSerivceToken,
  useClass: AiBackService
});

opts.injector = injector;
const serverApp = new ServerApp(opts);

自此,模块的引入阶段已经完成,不过这时候 AI 能力是缺失的(比如 inline chat 不触发等)

所以就需要在 AiNativeContribution 里提供各种 provider 的能力

贡献 Contribution

我们以 inline chat 为例,在 AiNativeContribution 中实现 registerInlineChatFeature 方法

@Domain(AINativeCoreContribution)
export class AiNativeContribution implements AINativeCoreContribution {
  // 通过 AIBackSerivcePath 拿到注册好的后端服务
  // 此时就能直接 RPC 调用后端服务提供的函数
  @Autowired(AIBackSerivcePath)
  private readonly aiBackService: IAIBackService;

  registerInlineChatFeature(registry: IInlineChatFeatureRegistry) {
    registry.registerEditorInlineChat(
      {
        id: 'ai-comments',
        name: 'Comments',
        title: 'add comments',
        renderType: 'button',
        codeAction: {}
      },
      {
        execute: async (editor: IEditor) => {
          // 在这里处理 inline chat 按钮点击之后的任何交互
        }
      }
    );

    registry.registerEditorInlineChat(
      {
        id: 'ai-optimize',
        name: 'Optimize',
        renderType: 'dropdown',
        codeAction: {}
      },
      {
        // 提供 diff 预览策略
        providerDiffPreviewStrategy: async (editor: ICodeEditor, token) => {
          const crossCode = this.getCrossCode(editor);
          const prompt = `Optimize the code:\n\`\`\`\n ${crossCode}\`\`\``;

          // 在这里通过调用后端服务的 request 方法拿到大模型返回的结果
          const result = await this.aiBackService.request(prompt, {}, token);

          if (result.isCancel) {
            return new CancelResponse();
          }

          if (result.errorCode !== 0) {
            return new ErrorResponse('');
          }

          return new ReplyResponse(result.data!);
        }
      }
    );
  }
}

此时选中编辑器内的某段代码,就会在合适的区域展示刚刚注册上去的 inline chat 按钮

其他能力的 Provider API 文档如下:

更详细完整的接口定义见:types.ts

方法名描述参数类型返回类型
registerInlineChatFeature注册 inline chat 相关功能IInlineChatFeatureRegistryvoid
registerChatFeature注册 chat 面板相关功能IChatFeatureRegistryvoid
registerChatRender注册 chat 面板相关渲染层,可以自定义 renderIChatRenderRegistryvoid
registerResolveConflictFeature注册智能解决冲突相关功能IResolveConflictRegistryvoid
registerRenameProvider注册智能重命名相关功能IRenameCandidatesProviderRegistryvoid
registerProblemFixFeature注册智能修复相关功能IProblemFixProviderRegistryvoid
registerIntelligentCompletionFeature注册智能代码补全相关功能IIntelligentCompletionsRegistryvoid

IInlineChatFeatureRegistry

方法名描述参数类型返回类型
registerEditorInlineChat注册编辑器中的 inline chat 功能AIActionItem, IEditorInlineChatHandlerIDisposable
registerTerminalInlineChat注册终端中的 inline chat 功能AIActionItem, ITerminalInlineChatHandlerIDisposable

IEditorInlineChatHandler

方法名描述参数类型返回类型
execute直接执行 action 的操作,点击后 inline chat 消失ICodeEditor, ...any[]void
providerDiffPreviewStrategy提供 diff editor 的预览策略ICodeEditor, CancellationTokenMaybePromise<ChatResponse>

ITerminalInlineChatHandler

方法名描述参数类型返回类型
triggerRules定义触发规则'selection' 或 BaseTerminalDetectionLineMatcher[]void
execute执行 action 的操作string (stdout), string (stdin), BaseTerminalDetectionLineMatcher? (rule)void

MaybePromise 是一个类型别名,表示方法可能返回一个 Promise 或者直接返回相应类型的值。

ChatResponse 是一个联合类型,表示响应可以是 ReplyResponseErrorResponseCancelResponse 中的任何一种。

export class ReplyResponse {
  constructor(readonly message: string) {}

  static is(response: any): boolean {
    return (
      response instanceof ReplyResponse ||
      (typeof response === 'object' && response.message !== undefined)
    );
  }
}

export class ErrorResponse {
  constructor(readonly error: any, readonly message?: string) {}

  static is(response: any): boolean {
    return (
      response instanceof ErrorResponse ||
      (typeof response === 'object' && response.error !== undefined)
    );
  }
}

export class CancelResponse {
  readonly cancellation: boolean = true;

  constructor(readonly message?: string) {}

  static is(response: any): boolean {
    return (
      response instanceof CancelResponse ||
      (typeof response === 'object' && response.cancellation !== undefined)
    );
  }
}

export type ChatResponse = ReplyResponse | ErrorResponse | CancelResponse;

用例:

// 注册编辑器内的 inline chat
registerInlineChatFeature(registry: IInlineChatFeatureRegistry) {
 registry.registerEditorInlineChat(
   {
     id: 'comments',
     name: 'Comments',
   },
   {
     providerDiffPreviewStrategy: async (editor: ICodeEditor, token) => {
       const crossCode = this.getCrossCode(editor);
       const prompt = `Comment the code: \`\`\`\n ${crossCode}\`\`\`. It is required to return only the code results without explanation.`;

       const result = await this.aiBackService.request(prompt, {}, token);

       if (result.isCancel) {
         return new CancelResponse();
       }

       if (result.errorCode !== 0) {
         return new ErrorResponse('');
       }

       return new ReplyResponse(result.data!);
     },
   },
 );
}

// 注册终端区域的 inline chat
registry.registerTerminalInlineChat(
   {
     id: 'terminal',
     name: 'terminal',
   },
   {
     triggerRules: 'selection',
     execute: async (stdout: string) => {},
   },
);

IChatFeatureRegistry

方法名描述参数类型返回类型
registerWelcome注册 Chat 面板的欢迎信息IChatWelcomeMessageContent 或 React.ReactNode, ISampleQuestionsvoid
registerSlashCommand注册 Chat 面板的快捷指令IChatSlashCommandItem, IChatSlashCommandHandlervoid

用例:

registerChatFeature(registry: IChatFeatureRegistry): void {
 registry.registerWelcome(
   new MarkdownString(`Hello, I am your dedicated AI assistant, here to answer questions about code and help you think. You can ask me some questions about code.`),
   [
     {
       icon: getIcon('send-hollow'),
       title: 'Generate a Java Quicksort Algorithm',
       message: 'Generate a Java Quicksort Algorithm',
     },
   ],
 );

 registry.registerSlashCommand(
   {
     name: 'Explain',
     description: 'Explain',
     isShortcut: true,
     tooltip: 'Explain',
   },
   {
     providerRender: CustomSlashCommand,
     providerInputPlaceholder(value, editor) {
       return 'Please enter or paste the code.';
     },
     providerPrompt(value, editor) {
       return `Explain code: \`\`\`\n${value}\n\`\`\``;
     },
     execute: (value: string, send: TChatSlashCommandSend, editor: ICodeEditor) => {
       send(value);
     },
   },
 );
}

IChatRenderRegistry

方法名描述参数类型返回类型
registerWelcomeRender自定义欢迎信息的视图渲染ChatWelcomeRendervoid
registerAIRoleRender自定义 AI 会话的视图渲染ChatAIRoleRendervoid
registerUserRoleRender自定义用户会话的视图渲染ChatUserRoleRendervoid
registerThinkingRender自定义 loading 时的视图渲染ChatThinkingRendervoid
registerInputRender自定义输入框的视图渲染ChatInputRendervoid

用例:

registerChatRender(registry: IChatRenderRegistry): void {
   // 直接注入 React 组件即可
   registry.registerWelcomeRender(CustomWelcomeComponents)
}

IResolveConflictRegistry

方法名描述参数类型返回类型
registerResolveConflictProvider注册解决冲突的模型问答服务keyof typeof MergeConflictEditorMode, IResolveConflictHandlervoid

用例:

registerResolveConflictFeature(registry: IResolveConflictRegistry): void {
 registry.registerResolveConflictProvider('traditional', {
   providerRequest: async (contentMetadata, options, token) => {
    return new ReplyResponse('Resolved successfully!')
   },
 });
}

IRenameCandidatesProviderRegistry

方法名描述参数类型返回类型
registerRenameSuggestionsProvider注册重命名建议的提供者NewSymbolNamesProviderFnvoid

用例:

registerRenameProvider(registry: IRenameCandidatesProviderRegistry): void {
  registry.registerRenameSuggestionsProvider(async (model, range, token): Promise<NewSymbolName[] | undefined> => {
    return {
     newSymbolName: 'ai rename',
     tags: [NewSymbolNameTag.AIGenerated],
    }
  });
}

IProblemFixProviderRegistry

方法名描述参数类型返回类型
registerHoverFixProvider注册问题诊断的提供者IHoverFixHandlervoid

用例:

registerProblemFixFeature(registry: IProblemFixProviderRegistry): void {
  registry.registerHoverFixProvider({
    provideFix: async (
      editor: ICodeEditor,
      context: IProblemFixContext,
      token: CancellationToken,
    ): Promise<ChatResponse | InlineChatController> => {
      const { marker, editRange } = context;
      const prompt = '可自行组装问题诊断的 prompt';

      const controller = new InlineChatController({ enableCodeblockRender: true });
      const stream = await this.aiBackService.requestStream(prompt, {}, token);
      controller.mountReadable(stream);

      return controller;
    },
  });
}

IIntelligentCompletionsRegistry

方法名描述参数类型返回类型
registerIntelligentCompletionProvider注册智能补全的提供者IIntelligentCompletionProvidervoid

说明: 在返回的补全列表当中配置 enableMultiLine 字段可开启多行补全的能力

用例:

registerIntelligentCompletionFeature(registry: IIntelligentCompletionsRegistry): void {
  registry.registerIntelligentCompletionProvider(async (editor, position, bean, token) => {
    const model = editor.getModel()!;
    const value = model.getValueInRange({
      startLineNumber: position.lineNumber,
      startColumn: 1,
      endLineNumber: position.lineNumber + 3,
      endColumn: model?.getLineMaxColumn(position.lineNumber + 3),
    });

    const cancelController = new AbortController();
    const { signal } = cancelController;

    token.onCancellationRequested(() => {
      cancelController.abort();
    });

    const getRandomString = (length) => {
      const characters = 'opensumi';
      let result = '';
      for (let i = 0; i < length; i++) {
        result += characters.charAt(Math.floor(Math.random() * characters.length));
      }
      return result;
    };

    /**
     * 随机增删字符
     */
    const insertRandomStrings = (originalString) => {
      const minChanges = 2;
      const maxChanges = 5;
      const changesCount = Math.floor(Math.random() * (maxChanges - minChanges + 1)) + minChanges;
      let modifiedString = originalString;
      for (let i = 0; i < changesCount; i++) {
        const randomIndex = Math.floor(Math.random() * originalString.length);
        const operation = Math.random() < 0.5 ? 'delete' : 'insert';
        if (operation === 'delete') {
          modifiedString = modifiedString.slice(0, randomIndex) + modifiedString.slice(randomIndex + 1);
        } else {
          const randomChar = getRandomString(1);
          modifiedString = modifiedString.slice(0, randomIndex) + randomChar + modifiedString.slice(randomIndex);
        }
      }
      return modifiedString;
    };

    try {
      await new Promise((resolve, reject) => {
        const timeout = setTimeout(resolve, 1000);

        signal.addEventListener('abort', () => {
          clearTimeout(timeout);
          reject(new DOMException('Aborted', 'AbortError'));
        });
      });

      return {
        items: [
          {
            insertText: insertRandomStrings(value),
            range: {
              startLineNumber: position.lineNumber,
              startColumn: 1,
              endLineNumber: position.lineNumber + 3,
              endColumn: model?.getLineMaxColumn(position.lineNumber + 3),
            },
          },
        ],
        // 是否启动多行补全
        enableMultiLine: true,
      };
    } catch (error) {
      if (error.name === 'AbortError') {
        return { items: [] };
      }
      throw error;
    }
  });
}

完整示例代码见 ai-native.contribution.ts

相关配置

AI Native Config 相关的配置参数可以控制所有 AI 能力的开关

IAINativeConfig

属性名类型描述
capabilitiesIAINativeCapabilitiesAI 功能的配置项
layoutIDesignLayoutConfig布局设计的配置项

IAINativeCapabilities

以下所有 capabilities 能力默认开启

属性名类型描述
supportsMarkersboolean是否开启 AI 能力处理问题面板
supportsChatAssistantboolean是否开启 AI 聊天助手功能
supportsInlineChatboolean是否开启 Inline Chat 功能
supportsInlineCompletionboolean是否开启代码智能补全功能
supportsConflictResolveboolean是否开启 AI 智能解决冲突的功能
supportsRenameSuggestionsboolean是否开启 AI 提供重命名建议的功能
supportsProblemFixboolean是否开启 AI 问题诊断能力
supportsTerminalDetectionboolean是否开启 AI 终端检测功能
supportsTerminalCommandSuggestboolean是否开启 AI 终端命令建议功能

IDesignLayoutConfig

属性名类型描述
useMergeRightWithLeftPanelboolean是否将右侧面板与左侧面板合并
useMenubarViewboolean是否使用新的菜单栏视图
menubarLogostring设置菜单栏的图标,可以是图片路径