HawkEye

Our project, HawkEye, is a Web3 and AI-powered solution designed to authenticate human gameplay in online gaming environments. In an era where bots and automated scripts undermine the integrity of gaming platforms, our solution provides a robust method to verify that an actual human is playing the game, enhancing trust and fairness in the gaming community.

Problem Statement: The rise of bots and automation in gaming has led to unfair advantages, disrupted in-game economies, and diminished player experiences. Traditional methods of bot detection are often reactive and can be circumvented. There's a pressing need for a proactive, reliable, and universally acceptable method to verify human players.

Solution Overview We address this challenge by capturing and analyzing keystroke data from two sources:

• Hardware Keystroke Capture: Directly records keystrokes from the player's hardware keyboard. Each key press is timestamped and stored in a CSV file.
• Video Stream Keystroke Detection: A mobile application records the player's gameplay video stream. The video is sent to a remote server where an AI model processes it to visually detect keystrokes. Detected keystrokes are also timestamped and stored in a CSV file.

Data Comparison using Dynamic Time Warping (DTW)

• Batch Processing: Keystroke data from both sources are batched over specified intervals to synchronize timestamps.
• DTW Algorithm: Applies the Dynamic Time Warping algorithm to compare the two sequences of keystrokes. This method effectively handles variations in timing and speed, which are natural in human interactions, and calculates a threshold value representing the similarity between the two datasets.
• Threshold Interpretation: A threshold value less than 5 indicates high similarity, confirming human gameplay. A value greater than or equal to 5 suggests discrepancies, potentially flagging non-human activity.

On-Chain Proof Generation

• Blockchain Network Integration: Once human gameplay is confirmed, the proof is generated and pushed onto the blockchain, leveraging its privacy-preserving and decentralized attributes.
• Universal Verification: The on-chain proof serves as a tamper-proof record. Players can present this proof to verify their human status across various gaming platforms.

Use Cases

• Competitive Gaming: Ensuring participants in tournaments are genuine players.
• In-Game Economy Protection: Preventing bots from farming resources and destabilizing virtual economies.
• User Onboarding: Verifying new users during account creation to reduce fraudulent activities.

Future Development Plans

• Real-Time Verification: Developing capabilities for continuous monitoring during gameplay sessions.
• Extended Biometric Analysis: Incorporating mouse movements, touch inputs, and other biometric data for enhanced verification.
• Platform SDK: Providing an SDK for game developers to integrate our verification system seamlessly.
• Community Engagement: Collaborating with gaming communities to promote fair play standards.

Conclusion: Our project aims to alter human verification in the gaming industry. Combining AI-driven keystroke detection with blockchain technology, we offer a novel solution that enhances security, fairness, and trust in online gaming. We think that HawkEye has the potential to become a standard for human verification across gaming platforms worldwide.

Keywords: Dynamic Time Warping, Human Verification, Gaming Security, Keystroke Detection, On-Chain Proof.

Our project, HawkEye, is a fusion of AI, blockchain, and mobile technologies, carefully staged to create a seamless and secure human verification system for online gaming. Here's a detailed breakdown of how we built it:

User behaviour monitoring (Frontend):

• HawkEye App: We created a flutter mobile application that records the user's keyboard input during gameplay by filming it. The app captures a video of the user's physical keystrokes as they play.
• Dart Language: As Flutter uses Dart, we leveraged it to write efficient UI code.

AI Model:

• Python: Selected to create scripts for keystroke data collection and running keystroke detection model due to its rich ecosystem in machine learning and AI.
• Pipeline: The video stream is broken down into frames. Each frame is analyzed using the AI model to detect visible keystrokes. The extracted inputs are compared with the user's in-game actions to check for consistency. Detected keystrokes are timestamped to match the hardware keystroke logs. By matching the physical keystrokes with the actions occurring in the game, we verify that a human is playing.

Blockchain Integration:

• Proof Generation: If human gameplay is confirmed, a proof object is created containing necessary verification details.
• Smart Contract Execution: The proof is pushed onto the blockchain network using smart contracts.
• On-Chain Storage: The proof is stored securely on-chain, ensuring immutability and privacy.
• Universal Verification: Players can present this on-chain proof across various gaming platforms to verify their human status.

Partner Technologies and Their Benefits:

• Phala Network: Their privacy-focused blockchain was integral to our project's requirement for secure and confidential proof storage. It allowed us to leverage Trusted Execution Environments (TEEs) to ensure that the data remains confidential and tamper-proof.

Some Innovative Aspects:

• Synchronization of Asynchronous Data Streams: Since video capture and keystroke logging operate at different frequencies and can experience latency, we developed a custom synchronization algorithm that aligns the timestamps of both data streams before applying DTW.
• Optimized DTW Implementation: Standard DTW can be computationally intensive. We optimized the algorithm by incorporating window constraints (Sakoe-Chiba band) to limit the warping path, significantly reducing computation time without sacrificing accuracy.
• Edge Computing Considerations: To reduce server load and latency, we explored moving some AI processing to the client side. Although not fully implemented, we created a prototype where the mobile app performs preliminary keystroke detection using TensorFlow Lite.