Fast Window Capture - OpenCV Object Detection in Games #4
Learn how to capture window data in real-time as a video stream for processing with OpenCV. We try several different methods searching for the fastest way possible. In this tutorial I also discuss the importance of good Google search skills as a programmer, and we revisit some basic object-oriented programming concepts.
Full tutorial playlist:
Grab the code on GitHub:
Research discussed:
OpenCV getting started with videos:
Fastest way to take a screenshot with Python on Windows:
Convert PyAutoGUI image to OpenCV:
Convert SaveBitmapFile to an OpenCV image instead:
Best Numpy reference:
List all your windows:
0:47 Main capture loop
3:16 Using PyAutoGUI screenshot()
6:24 Measure FPS
8:27 Using Pillow ImageGrab
9:31 Using Pywin32 for screenshots
13:48 Converting () for OpenCV
17:23 Confining screenshots to a specific window
18:42 Creating a WindowCapture class
25:05 Trimming the window capture
28:15 Image to screen position conversion
29:35 Wrap up
Read the full written tutorial with code samples here:
Up to this point, we’ve been using OpenCV to detect objects in static images. Now we’re ready to apply those same techniques to video games in real time.
Remember that video is just a series of still images shown in rapid succession. In this tutorial our goal is to capture screenshots as fast as possible and display them in an OpenCV window, so that we get a real time video stream of the game we’re interested in. This will set us up to begin processing image data with OpenCV in real-time.
OpenCV has a tutorial on “Getting Started with Videos“ that will serve as the basis for our code. Our starting point differs from the official tutorial only in that we are preparing to work with screenshot data instead of frames from a camera.
When defining get_screenshot() you could simply use () from the PyAutoGUI library, or () from the Python Image Library.
And this would work, but there are several benefits to calling the Windows API directly instead. Firstly, we approach the theoretical limit for how fast we can take these screenshots by dealing right with the operating system itself. Secondly, the Windows API has methods that will allow us to grab the screen data for only the window we’re interested in, even when it’s minimized or off screen.
To do this, we must first pip install pywin32 to get access to the Win32 API in Python.
Let’s start with some code to capture a single screenshot of our entire desktop and save that to a file. This will confirm for us that the Windows API calls are working. By calling this function, you should end up with a screenshot file.
The next step is to modify this function so that instead of saving an image file, it instead returns the image data, formatted to work with OpenCV.
Now we can call this function from our original infinite loop and get a real-time stream of our desktop.
To improve upon this, we can use (None, window_name) to capture just the window we’re interested in. Replace the window_name with a string that contains the name found in the title bar of the window you want to capture. Doing so will allow you to capture the frames from that window even when it’s hidden behind other windows.
If you’re having trouble figuring out the name of the window you want, you can use this code to list the names of all your existing windows:
We can improve our code further by trimming off the excess around the window we’re interested in. When you run the above code, you will notice black space to the right and below the window image, as well as the window borders and title bar. Removing these will not only clean things up, it will also improve our frame rate. We can also get improvements by not calling () on every call to get_screenshot(), so let’s turn this into a class.
Finally, we’ll need a way to convert positions we detect in our screenshots back to pixel positions on our actual monitor. In the WindowCapture class constructor, I’ve already included code to calculate the window offset using the window position data from (). Let’s add a method to our class that uses this offset to return that converted screen position.
Continue with the written tutorial here:
1 view
384
115
11 hours ago 00:03:22 1
The Fall Guy | Official Trailer
14 hours ago 00:00:00 1
WoT Blitz. Late Night Birthday Lotto + Gifts and Presents
2 days ago 00:04:05 1
Learn the English Phrases IT’S REALLY COMING DOWN OUT THERE and RAINING CATS AND DOGS
3 days ago 00:02:07 8
It’s all WoT Blitz! But it’s not all...
3 days ago 00:11:04 1
Gnome Boxes | Installing Windows 10 on Linux
4 days ago 00:05:10 1
DevLog#01 - Sims like building, walls, floor, windows and doors in Unreal Engine 4
4 days ago 00:14:12 2
“ПЫЛИ НЕТ!“ Видеоинструкция по доработке москитной сетки! В РАЗЫ УМЕНЬШАЕМ КОЛИЧЕСТВО ПЕСКА С УЛИЦЫ.
6 days ago 00:14:21 2
2024 Surface Laptop 7th Gen - Unboxing, Setup and First Look
6 days ago 00:16:32 1
I got this PC from a fast food restaurant. What now?
6 days ago 00:00:25 1
🎁 Grab your gift in WoT Blitz until June 30. All you need is just to log in the game!
7 days ago 00:31:14 1
RTX 4070 + i5 13400F : Test in 18 Games - RTX 4070 Gaming Test
7 days ago 00:32:46 1
RTX 3070 + i5 12400F : Test in 18 Games - RTX 3070 Gaming
7 days ago 00:12:27 1
RTX 3070 + i5 12400F : Test in 13 Games - RTX 3070 Gaming
1 week ago 00:08:21 1
Brimstone Lineups Haven 2024 *NEW*
1 week ago 00:05:20 1
TLC - Waterfalls (Official HD Video)
1 week ago 00:04:26 1
Stocking Strategy Part 1 - Picking Fish for Your Saltwater Aquarium - Livestock Series
1 week ago 00:17:49 1
BYD Yangwang U8 (2024) - China’s Most Luxurious and Craziest SUV Ever!