OSWorld:在真实计算机环境中为开放式任务进行多模态代理基准测试

参考

Abstract(摘要)

Autonomous agents that accomplish complex computer tasks with minimal human interventions have the potential to transform human-computer interaction, significantly enhancing accessibility and productivity. However, existing benchmarks either lack an interactive environment or are limited to environments specific to certain applications or domains, failing to reflect the diverse and complex nature of real-world computer use, thereby limiting the scope of tasks and agent scalability.

UI-TARS: Pioneering Automated GUI Interaction with Native Agents

UI-TARS: Pioneering Automated GUI Interaction with Native Agents(与本地代理进行自动化 GUI 交互的先驱)

Abstract(摘要)

This paper introduces UI-TARS, a native GUI agent model that solely perceives the screenshots as input and performs human-like interactions (e.g., keyboard and mouse operations). Unlike prevailing agent frameworks that depend on heavily wrapped commercial models (e.g., GPT-4o) with expert-crafted prompts and workflows, UI-TARS is an end-to-end model that outperforms these sophisticated frameworks.

CUA 评估额外信息

CUA eval extra information

This document includes extra information to how we evaluated our Computer Using Agent, including (browser/VM) environments, prompts, sampling parameters, and scoring procedures. For more details, read https://openai.com/index/computer-using-agent/.

本文档包括我们如何评估我们的计算机使用代理的额外信息,包括(浏览器/VM)环境,提示,采样参数和评分程序。有关更多详细信息,请阅读 https://openai.com/index/computer-using-agent/

1 Environment(环境)

  • For WebArena and WebVoyager, we run the evals in operator browser instead of playwright browsers since our model relies on the visual action space for navigation (search bar, backward/forward button). Our model does not have access to tool calls that control the navigation.
  • 对于WebArena和WebVoyager,我们在 operator browser 中运行评估,而不是在 playwright 浏览器中运行,因为我们的模型依赖于用于导航的视觉动作空间(搜索栏,后退/前进按钮)。我们的模型无法访问控制导航的工具调用。
  • For OSWorld, we use the VMWare Ubuntu VM distributed by the authors. Our environment has the dock on the right side of the screen instead of the left side, which we have found to improve the performance slightly.
  • 对于 OSWorld,我们使用作者分发的 VMWare Ubuntu VM。我们的环境将 dock 放在屏幕的右侧,而不是左侧,我们发现这样可以稍微提高性能。

Computer-Using Agent

Computer-Using Agent (CUA)

A universal interface for AI to interact with the digital world. AI 与数字世界交互的通用接口。

Today we introduced a research preview of Operator⁠, an agent that can go to the web to perform tasks for you. Powering Operator is Computer-Using Agent (CUA), a model that combines GPT-4o's vision capabilities with advanced reasoning through reinforcement learning. CUA is trained to interact with graphical user interfaces (GUIs)—the buttons, menus, and text fields people see on a screen—just as humans do.

Operator System Card

1 Introduction(简介)

Operator is a research preview of our Computer-Using Agent (CUA) model, which combines GPT-4o’s vision capabilities with advanced reasoning through reinforcement learning. It interprets screenshots and interacts with graphical user interfaces (GUIs) — the buttons, menus, and text fields people see on a computer screen — just as people do. Operator’s ability to use a computer enables it to interact with the same tools and interfaces that people rely on daily, unlocking the potential to assist with an unparalleled range of tasks.

Operator 是我们计算机使用代理(CUA)模型的研究

DeepSeek-V3 Technical Report

Abstract(摘要)

We present DeepSeek-V3, a strong Mixture-of-Experts (MoE) language model with 671B total parameters with 37B activated for each token. To achieve efficient inference and cost-effective training, DeepSeek-V3 adopts Multi-head Latent Attention (MLA) and DeepSeekMoE architec- tures, which were thoroughly validated in DeepSeek-V2. Furthermore, DeepSeek-V3 pioneers an auxiliary-loss-free strategy for load balancing and sets a multi-token prediction training objective for stronger performance. We pre-train DeepSeek-V3 on 14.

DeepSeek R1: 通过强化学习激励 LLM 的推理能力

Abstract(摘要)

We introduce our first-generation reasoning models, DeepSeek-R1-Zero and DeepSeek-R1. DeepSeek-R1-Zero, a model trained via large-scale reinforcement learning (RL) without super- vised fine-tuning (SFT) as a preliminary step, demonstrates remarkable reasoning capabilities. Through RL, DeepSeek-R1-Zero naturally emerges with numerous powerful and intriguing reasoning behaviors. However, it encounters challenges such as poor readability, and language mixing.

CodeGate - 让 AI 编码助手更安全

什么是 CodeGate

CodeGate 是位于 AI 编码助手和 LLM 之间的本地提示网关,用于增强隐私和安全性。

  • 执行代码安全审查
  • 识别包依赖项中的漏洞
  • 防止敏感数据(如机密)与 AI 模型共享

工作原理

CodeGate 是位于 AI 编码助手和 LLM 之间的本地代理。CodeGate 会审查您的提示是否存在任何潜在的机密泄露 — 在机密离开您的桌面之前对其进行加密,并在响应中对其进行解密。CodeGate 使用 RAG 来更新任何 LLM 的知识库,并提供相关的风险洞察。

Continue 指南

启动 CodeGate 服务

docker pull ghcr.io/stacklok/codegate:latest
docker run --name codegate -d -p 8989:8989 -p 9090:9090 --restart unless-stopped ghcr.io/stacklok/codegate:latest

下载 Ollama 代码模型

ollama pull qwen2.5-coder:7b
ollama pull qwen2.5-coder:1.5b

配置 Continue 扩展

编辑配置文件:~/.continue/config.json

腾讯会议中云录制的 AI+

腾讯会议中云录制应用的核心:

  • 快速定位(章节、发言人、话题)
  • 转写、纪要、总结
  • 内容问答(AI小助手)

视频的主界面

下方区域

总结

章节

发言人

话题

右侧区域

转写

可以使用句子进行视频定位

纪要

可以按 章节主题发言人 进行纪要生成。

会议待办

AI小助手

这个AI小助手价格太贵了,可能对于中大型企业用户有一定吸引力,没有多少录制视频的用户基本不用考虑,上面的功能已经足够了。

这个营销太差了,这么高的价格也不给人试用,上来就收费,打击用户积极性。

下面是AI会议中提供的AI小助手

如何投资个人养老金

2024年12月15日,个人养老金制度正式在全国全面实施。这里记录一下如何投资个人养老金。

个人养老金

个人养老金制度全面实施

指数基金

什么是个人养老金

基金数据

2024年 第三季度 基金管理机构非货币理财公募基金月均规模排名

排名 公募基金管理人名称 非货币理财公募基金 月均规模(亿元) 排名 公募基金管理人名称 非货币理财公募基金 月均规模(亿元)
1 易方达基金管理有限公司 12307 11 鹏华基金管理有限公司 4225
2 华夏基金管理有限公司 10557 12 景顺长城基金管理有限公司 3881
3 广发基金管理有限公司 7887 13 工银瑞信基金管理有限公司 3695
4 嘉实基金管理有限公司 6598 14 国泰基金管理有限公司 3437
5 富国基金管理有限公司 6105 15 天弘基金管理有限公司 3431
6 南方基金管理股份有限公司 5945 16 华安基金管理有限公司 3419
7 博时基金管理有限公司 5677 17 永赢基金管理有限公司 3202
8 招商基金管理有限公司 5504 18 中银基金管理有限公司 3135
9 华泰柏瑞基金管理有限公司 4878 19 中欧基金管理有限公司 2869
10 汇添富基金管理股份有限公司 4796 20 兴证全球基金管理有限公司 2648

剔除了短期理财债券基金规模和基金中基金持有的自身管理的基