How to Extract Hardcoded Subtitles from Video with a Combination of VideoSubFinder and ABBYY FineReader?

VideoSubFinder is a free program that allows you to autodetect a video frame by frame and extract hardcoded subtitles to a series of image grabs with text based on text mining algorithms for further OCR process. Closely follow the steps below.

Step 1. Download and install VideoSubFinder here: https://sourceforge.net/projects/videosubfinder/. Since this program requires "Microsoft Visual C++ Redistributable for Visual Studio 2015, 2017 and 2019" installed on the PC, to avoid any error message, we have to install it beforehand via https://support.microsoft.com/en-us/help/2977003/the-latest-supported-visual-c-downloads.

Download and Install VideoSubFinder

Step 2. Run "VideoSubFinderWXW.exe", click on "File" and select "Open Video (OpenCV)" to import the video file embedded with hardsubs. As an aside, this program supports major video formats, however, if your video file type is not supported, try to convert it with some free video converter programs.

Import Video to VideoSubFinder

Step 3. After importing the video file, drag the slider along the progress bar to locate the subtitle position (it generally displays on the bottom). To precisely frame the area where the subtitle appears in the video, drag down the tiny black sliders as shown in the screenshot below to eliminate redundant video screen part. Then press "Run Search" button to autodetect the hardsubs.

Extract Hardcoded Subtitles from MP4

Step 4. When the process is finished, switch to OCR tab and click on "Create Cleared TXT Images". After done, it will produce a number of large cleared image sequences with text in the TXTImages folder of VideoSubFinder root directory, e.g. "E:\VideoSubFinder_5.10_x64\Release_x64\TXTImages". DO NOT close the program and go to the next step.

Generate Cleared TXT Images

Step 5. At this point, we have to use some image to text recognition software for OCR process. Lucky me, I have purchased a subscription of ABBYY FineReader for years because of its powerful AI-based OCR technology. I'll take it as an example, and you can also find some free OCR software to go ahead with the following steps. Run FineReader, click on "Convert to Other Formats", go to "TXTImages" folder and select all generated images to import to FineReader.

OCR Hardcoded Subtitles

Step 6. Select output format as TXT Document and desired OCR language. Hit "Convert to TXT" button and specify "TXTResults" folder as the output folder. Then wait for the images to texts conversion to be complete. Based on the amount of images, it may need a while. Just go for a coffee break!

OCR Subtitles from Video

Step 7. Now the "TXTResults" folder is full of OCR-ed .txt files. Go back to VideoSubFinder, hit "Create Sub From TXT Results" button to generate a .srt (recommended) or .ass subtitle file. Rename the subtitle file and save it.

Create Sub from Converted Results

Step 8. You can open the subtitle with any text editor like Notepad. That's the end of the tutorial. But if you're not pleased the extracted hardcoded subtitle file, you can also take an optional step to fine tune and sync the SRT file with some subtitle editors, such as Aegisub, Subtitle Adjuster, etc.