qaboard 0.8.3

Visualize and compare algorithm results. Optimize parameters. Share results and track progress.

QA-Board

QA-Board helps Algorithms/QA engineers build great products with powerful quality evaluation and collaboration tools.

QA-Board is not officially released yet, and likely won't work yet for you because it expects running on our infra...

We're working on it! Status: https://github.com/Samsung/qaboard/issues/1

Features

• Organize, View and Compare Results, Tuning/Optimization
• Web-based: sharable URLs, no install needed.
• Visualizations: support for quantitative metrics, and many file formats: advanced image viewer, support for videos, plotly graphs, text, pointclouds, HTML...
• Integrations: direct access from Git and CI tools, easily exportable results, API, links to the code, trigger jobs...
• Agnostic to your language/framework: run your existing code, write files, view them.

Benefits

• Scale R&D: enable engineers to achieve more and be more productive.
• Faster Time-to-Market: collaboration across teams, workflow integration..
• Quality: uncover issues earlier, KPIs, tuning, reporting...

Get in touch

We are looking for feedback and insights from outside Samsung. This will help us set the direction for qaboard.

We think you could be interested if have projects where:

• unit tests are not enough (ML, operational research...), and a loss function doesn't tell the whole story. Maybe because...
• there are performance / quality trade-offs, or different configurations of your code you need to compare (hardware design, mobile/embedded...).
• you need advanced visualizations to make sense of results (statistics, image processing, 3d sensors, sensing and decision tasks...) and need tools to dive down into outputs.
• lifecycles are complex, with many stakeholders (algo, hardware, software, QA, production...)

Contact [e-mail]

Getting Started

Read the docs! You will learn how to:

• install QA-Board's CLI wrapper
• run a QA-Board server
• wrap your code with QA-Board
• view output files and KPIs
• ...and improve your integration with many guides: bit-accuracy, tuning, etc.

Code organization

Each section has its own README:

• qaboard: provides the qa CLI wrapper than runs your code, and the import qaboard package.
• webapp is the frontend that displays results.
• backend exposes an HTTP API used to read/write all the metadata on runs.
• webapp is the frontend that displays results.
• website is the QA-Board website and docs - samsung.github.io/qaboard
• thirdparty:
  • Cantaloupe IIIF server, used to "stream" large images to the users.

Questions? Need Help? Found a bug?

If you've got questions about setup, deploying, want to develop new features, or just want to chat with the developers, please feel free to start a thread in our Spectrum community!

Found a bug with QA-Board? Go ahead and submit an issue. And, of course, feel free to submit pull requests with bug fixes or changes to the master branch.

Contributors

QA-Board was started at Samsung SIRC by Arthur Flam.

Thanks to the following people for their contributions, testing, feedback or bug reports: Amir Fruchtman, Avi Schori, Yochay Doutsh, Itamar Persi, Amichay Amitay, Lena Grechikhin, Shai Shamir, Matan Danino, Roy Shaul, Gal Hai, Rivka Emmanuel, Nadav Ofer. Thanks also to Sebastien Derhy, Elad Rozin, Nathan Levy, Shahaf Duenyas, Yotam Ater, Asaf Jazcilevich and Yoel Yaffe for supporting the project.

You don't see your name? Get in touch to be added to the list!

Credits

• The logo is a the Poodle twemoji 🐩, recolored in Samsung Blue 🔵. Copyright 2019 Twitter, Inc and other contributors. Code licensed under the MIT License. Graphics licensed under CC-BY 4.0