Skill UI

This comprehensive benchmark suite evaluates the performance of Vision-Language Models (VLM) on complex video anomaly detection tasks using the SmartHome-Bench dataset. It covers a wide range of real-world smart home scenarios, including security breaches, elderly fall detection, and unusual pet behavior. Unlike single-frame analysis, this benchmark requires deep multi-frame video understanding, making it a robust tool for testing the advanced AI capabilities of video monitoring systems.

This guide provides a comprehensive tutorial on running various AI inference tasks using the Together AI API, which is compatible with the OpenAI standard. It demonstrates key functionalities including chat completions, streaming responses, image generation, and creating embeddings. This is ideal for developers who need to test open-source models, compare performance across different LLMs (like Llama or Mixtral), or integrate powerful generative AI features into their applications using Python or Node.js.

This skill enables the execution of OpenSquilla agents against complex, real-world software engineering benchmarks (SWE-bench). It runs the entire process—from pulling the necessary Docker image to collecting the patch and running official evaluations—in a containerized environment. Use it to test an agent's ability to solve specific coding issues, benchmark performance across datasets (e.g., verified, multilingual), or verify if a generated patch successfully resolves an instance.

This skill enables systematic improvement of existing AI agents by analyzing performance metrics, identifying failure modes, and applying prompt engineering techniques. It involves establishing baselines, optimizing prompts through chain-of-thought and few-shot examples, refining role definitions, and validating changes through structured testing and A/B comparisons to ensure reliability and efficiency.

Provides comprehensive guidance on building robust evaluation frameworks for complex agent systems. It addresses the inherent challenges of non-determinism and multiple valid paths, enabling systematic performance testing, validating context engineering choices, and measuring continuous improvements across multiple quality dimensions (e.g., factual accuracy, tool efficiency).