Cloudbank is built to be a Computer Vision and AI training platform alternative to its inspiration “OpenAI Universe” where AI agents can be trained to complete the task within a video game environment on Steam.
The Computer Vision part is handled using OpenCV 3 (C++) while it is planned for the Machine Learning part to be handled using an ML library in Python. Gstreamer will be used to capture the image frame and a bash script will be used to send simulated keyboard and mouse clicks to the Steam window currently playing the game.
Below is a demo video:
For more information and source code please visit the projects GitHub page
Sheuns blog 