Apple Vision framework – Text extraction from image

Question

I am using Vision framework for iOS 11 to detect text on image.

The texts are getting detected successfully, but how we can get the detected text?

Answer 1

This will return a overlay image with rectangle box on detected text

Here is the full xcode project https://github.com/cyruslok/iOS11-Vision-Framework-Demo

Hope it is helpful

// Text Detect
func textDetect(dectect_image:UIImage, display_image_view:UIImageView)->UIImage{
    let handler:VNImageRequestHandler = VNImageRequestHandler.init(cgImage: (dectect_image.cgImage)!)
    var result_img:UIImage = UIImage.init();

    let request:VNDetectTextRectanglesRequest = VNDetectTextRectanglesRequest.init(completionHandler: { (request, error) in
        if( (error) != nil){
            print("Got Error In Run Text Dectect Request");

        }else{
            result_img = self.drawRectangleForTextDectect(image: dectect_image,results: request.results as! Array<VNTextObservation>)
        }
    })
    request.reportCharacterBoxes = true
    do {
        try handler.perform([request])
        return result_img;
    } catch {
        return result_img;
    }
}

func drawRectangleForTextDectect(image: UIImage, results:Array<VNTextObservation>) -> UIImage {
    let renderer = UIGraphicsImageRenderer(size: image.size)
    var t:CGAffineTransform = CGAffineTransform.identity;
    t = t.scaledBy( x: image.size.width, y: -image.size.height);
    t = t.translatedBy(x: 0, y: -1 );

    let img = renderer.image { ctx in
        for item in results {
            let TextObservation:VNTextObservation = item
            ctx.cgContext.setFillColor(UIColor.clear.cgColor)
            ctx.cgContext.setStrokeColor(UIColor.green.cgColor)
            ctx.cgContext.setLineWidth(1)
            ctx.cgContext.addRect(item.boundingBox.applying(t))
            ctx.cgContext.drawPath(using: .fillStroke)

            for item_2 in TextObservation.characterBoxes!{
                let RectangleObservation:VNRectangleObservation = item_2
                ctx.cgContext.setFillColor(UIColor.clear.cgColor)
                ctx.cgContext.setStrokeColor(UIColor.red.cgColor)
                ctx.cgContext.setLineWidth(1)
                ctx.cgContext.addRect(RectangleObservation.boundingBox.applying(t))
                ctx.cgContext.drawPath(using: .fillStroke)
            }
        }

    }
    return img
}

Answer 2

Not exactly a dupe but similar to: Converting a Vision VNTextObservation to a String

You need to either use CoreML or another library to perform OCR (SwiftOCR, etc.)

Answer 3

In Apple Vision you can easily extract text from image using VNRecognizeTextRequest class, allowing you to make an image analysis request that finds and recognizes text in an image.

VNRecognizeTextRequest works starting from iOS 13.0 and macOS 10.15.

Here's a code snippet showing you how to do it:

let requestHandler = VNImageRequestHandler(url: imageURL, options: [:])

let request = VNRecognizeTextRequest { (request, error) in

    guard let observations = request.results as? [VNRecognizedTextObservation] 
    else { return }

    for observation in observations {

        let topCandidate: [VNRecognizedText] = observation.topCandidates(1)

        if let recognizedText: VNRecognizedText = topCandidate.first {
            label.text = recognizedText.string
        }
    }
}

Then you have to assign a value for recognitionLevel instance property:

// non-realtime asynchronous but accurate text recognition
request.recognitionLevel = VNRequestTextRecognitionLevel.accurate

// nearly realtime but not-accurate text recognition
request.recognitionLevel = VNRequestTextRecognitionLevel.fast

try? requestHandler.perform([request])