HackerEarth EmpowerX Challenge

Problem Statement

Develop a tool, bot, or extension for real-time ASL interpretation and closed captioning in Microsoft Teams.

HackerEarth is organizing a worldwide hackathon to develop a tool, bot, or extension that enhances accessibility in Microsoft Teams meetings. This tool must provide real-time American Sign Language (ASL) interpretation and closed captioning in U.S. English. It must use avatars or gravatars to represent ASL gestures and include features that allow users to activate these functions on demand. The tool should be designed with configurable settings to cater to various user needs and integrate with Microsoft Teams, utilizing Microsoft access mechanisms.

Tech Stack

Open to all

Examples for your reference only:

• Programming Language: Python, TypeScript, Java, JavaScript, or C#
• APIs: Azure OpenAI service - RestAPI references, Microsoft Graph API
• Platforms: Microsoft Teams, Microsoft Azure
• Frameworks/Libraries: Microsoft Bot Framework SDK
• Tools: Avatars and Gravatars

Tasks

Business Requirements for the Project:

For Meeting Participants and Meeting Owners:

• Display ASL in real-time in Microsoft Teams using avatars/gravatars on demand.
• Support real-time closed captioning in Microsoft Teams alongside ASL (American Sign Language).
• Display secure messages to end-users.
• Ensure your solution complies with security guidelines, including Section 508 and ADA standards.

For Tool Admins and Meeting Owners:

• Allow the meeting organizer to enable or disable the ASL and closed captioning features as needed.

Technical Requirements

Security Compliance:

• Ensure the tool complies with OWASP and NIST security standards, including all vendor libraries and packages.
• The tool must be free from any security vulnerabilities.

Deployment:

• The tool must be configurable and deployable within a HackerEarth-approved environment.
• Integration must utilize Azure OpenAI REST API calls, which are pre-approved for use.
• The production environment must exclusively use large language models (LLMs) that are approved by HackerEarth.
• No data should be stored outside the approved ecosystem.
• Settings for REST API calls and avatar/gravatar locations should be adjustable through configuration files, without needing to change the underlying code.
• Implement effective mechanisms to handle exceptions and errors.
• Ensure the tool is secure and free from any security vulnerabilities.

Deliverables

• Development and Integration: Provide details on the complete, functional tool or bot and its integration. (Mandatory for idea submission) [Idea Submission Template (for reference)]
• Documentation: Submit clear documentation for code, setup, and deployment to ensure smooth integration into the approved ecosystem. Show that the tool can be easily set up and used to meet all specified requirements within the development environment.
• Demo: Submit a video and link to the project (Mandatory for prototype).