Media is too big
VIEW IN TELEGRAM
https://os-world.github.io/ #гис
Autonomous agents that accomplish complex computer tasks with minimal human interventions has the potential to transform human-computer interaction, significantly enhancing accessibility and productivity. However, existing benchmarks either lack an interactive environment or are limited to environments specific to certain applications or domains, failing to reflect the diverse and complex nature of real-world computer use, thereby limiting the scope of tasks and agent scalability. To address this issue, we introduce OSWorld, the first-of-its-kind scalable, real computer environment for multimodal agents, supporting task setup, execution-based evaluation, and interactive learning across various operating systems such as Ubuntu, Windows, and macOS. OSWorld can serve as a unified, integrated computer environment for assessing open-ended computer tasks that involve arbitrary applications. Building upon OSWorld, we create a benchmark of 369 computer tasks involving real web and desktop apps in open domains, OS file I/O, and workflows spanning multiple applications. Each task example is derived from real-world computer use cases and includes a detailed initial state setup configuration and a custom execution-based evaluation script for reliable, reproducible evaluation. Extensive evaluation of state-of-the-art LLM/VLM-based agents on OSWorld reveals significant deficiencies in their ability to serve as computer assistants. While humans can accomplish over 72.36% of the tasks, the best model achieves only 12.24% success, primarily struggling with GUI grounding and operational knowledge. Comprehensive analysis using OSWorld provides valuable insights for developing multimodal generalist agents that were not possible with previous benchmarks.
Autonomous agents that accomplish complex computer tasks with minimal human interventions has the potential to transform human-computer interaction, significantly enhancing accessibility and productivity. However, existing benchmarks either lack an interactive environment or are limited to environments specific to certain applications or domains, failing to reflect the diverse and complex nature of real-world computer use, thereby limiting the scope of tasks and agent scalability. To address this issue, we introduce OSWorld, the first-of-its-kind scalable, real computer environment for multimodal agents, supporting task setup, execution-based evaluation, and interactive learning across various operating systems such as Ubuntu, Windows, and macOS. OSWorld can serve as a unified, integrated computer environment for assessing open-ended computer tasks that involve arbitrary applications. Building upon OSWorld, we create a benchmark of 369 computer tasks involving real web and desktop apps in open domains, OS file I/O, and workflows spanning multiple applications. Each task example is derived from real-world computer use cases and includes a detailed initial state setup configuration and a custom execution-based evaluation script for reliable, reproducible evaluation. Extensive evaluation of state-of-the-art LLM/VLM-based agents on OSWorld reveals significant deficiencies in their ability to serve as computer assistants. While humans can accomplish over 72.36% of the tasks, the best model achieves only 12.24% success, primarily struggling with GUI grounding and operational knowledge. Comprehensive analysis using OSWorld provides valuable insights for developing multimodal generalist agents that were not possible with previous benchmarks.
Блог инженера Шмакова
https://os-world.github.io/ #гис Autonomous agents that accomplish complex computer tasks with minimal human interventions has the potential to transform human-computer interaction, significantly enhancing accessibility and productivity. However, existing…
В то время как люди могут выполнить более 72,36% задач, лучшая модель достигает успеха только в 12,24%, в основном из-за проблем с основами графического пользовательского интерфейса и эксплуатационными знаниями. Комплексный анализ с использованием OSWorld предоставляет ценную информацию для разработки мультимодальных универсальных агентов, что было невозможно с помощью предыдущих тестов.
Еще одно "доказательство" в сторону создания гиперконвергентного мультимодального контекстно-ориентированного пользовательского интерфейса общего назначения #гис
Блог инженера Шмакова
#гис https://www.youtube.com/watch?v=2En8cj6xlv4
YouTube
WebAssembly at Google by Thomas Steiner & Thomas Nattestadt @ Wasm I/O 2024
Wasm I/O 2024 / 14-15 March 2024
Slides: https://goo.gle/wasmio24
From the V8 team to the Emscripten toolchain team to the many product teams that benefit from the advantages of Wasm in their libraries and flagship apps, WebAssembly plays a crucial role…
Slides: https://goo.gle/wasmio24
From the V8 team to the Emscripten toolchain team to the many product teams that benefit from the advantages of Wasm in their libraries and flagship apps, WebAssembly plays a crucial role…
Forwarded from Бугагашеньки 😁
Чертёж в хату, ГОСТ в радость, господа инженеры!
Forwarded from Российская Газета | Новости
Media is too big
VIEW IN TELEGRAM
Любопытная белуха присоединилась к компании сапсерферов во Владивостоке
Встретиться с игривой белухой посчастливилось жителям Владивостока. Произошло это в проливе Старка, который разделяет острова Русский и Попова.
В ноябре 2019 года 90 белух выпустили из бухты Средняя под Находкой. Некоторые из них поселились у берегов Приморья в Амурском заливе. Животные адаптировались в водах Японского моря и регулярно проявляют интерес к людям.
@rgrunews
Встретиться с игривой белухой посчастливилось жителям Владивостока. Произошло это в проливе Старка, который разделяет острова Русский и Попова.
В ноябре 2019 года 90 белух выпустили из бухты Средняя под Находкой. Некоторые из них поселились у берегов Приморья в Амурском заливе. Животные адаптировались в водах Японского моря и регулярно проявляют интерес к людям.
@rgrunews