Ask Ubuntu is a question and answer site for Ubuntu users and developers. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

In Ubuntu 12.10, if I type

gnome-screenshot -a | tesseract output

it returns:

** Message: Unable to use GNOME Shell's builtin screenshot interface, resorting to fallback X11.

How can I select a text from the screen and convert it to text (clipboard or document)?

Thank you!

share|improve this question
+1, this is a very interesting question! – Glutanimate Apr 11 '13 at 22:33
You get that error using only gnome-screenshot -a? Also why you pipe the output to tesseract? If i'm not wrong gnome-screenshot saves the picture on a file, and does not "print" it... – Salem Apr 12 '13 at 14:04
Warning should be harmless if I look through bugzilla. Question: what is the auto-save-directory? And did it drop anything in there? Interesting link: – Rinzwind Apr 12 '13 at 14:04
gnome-screenchot -a -c is supposed to copy selection to clipboard,isnt it?. but piping it to tesseract gives the same error. the default directory is home/pictures (works well). – Erling Apr 12 '13 at 14:39
Just done this using gnome-screenshot - I then had to edit the files to decrease the colour depth from 16m to 2 (it was black text on white background, but with today's fancy font smoothing and so-on, it wasn't really black) I then had to scale the image up to 200% of the original before I got an accurate OCR from tesseract - but it worked really well once I'd done that. – user192591 Sep 13 '13 at 11:01
up vote 17 down vote accepted

Maybe there is already some tool that does that, but you can also create a simple script with some screenshot tool and tesseract, as you are trying to use.

Take as an example this script (in my system I saved it as /usr/local/bin/screen_ts):

# Dependencies: tesseract-ocr imagemagick scrot

trap "rm $SCR_IMG*" EXIT

scrot -s $SCR_IMG.pngOOO
mogrify -modulate 100,0 -resize 400% $SCR_IMG.png #should increase detection rate
tesseract $SCR_IMG.png $SCR_IMG &> /dev/null
cat $SCR_IMG.txt


And with clipboard support:

# Dependencies: tesseract-ocr imagemagick scrot xsel

trap "rm $SCR_IMG*" EXIT

scrot -s $SCR_IMG.png
mogrify -modulate 100,0 -resize 400% $SCR_IMG.png #should increase detection rate
tesseract $SCR_IMG.png $SCR_IMG &> /dev/null
cat $SCR_IMG.txt | xsel -bi


It uses scrot to take the screen, tesseract to recognize the text and cat to display the result. The clipboard version additionally utilizes xsel to pipe the output into the clipboard.

sample usage

NOTE: scrot, xsel, imagemagick and tesseract-ocr are not installed by default but are available from the the default repositories.

You may be able to replace scrot with gnome-screenshot, but it may take a lot of work. Regarding the output you can use anything that can read a text file (open with Text Editor, show the recognized text as a notification, etc).

GUI version of the script

Here's a simple graphical version of the OCR script including a language selection dialog:

# DEPENDENCIES: tesseract-ocr imagemagick scrot yad
# AUTHOR:       Glutanimate 2013 (
# NAME:         ScreenOCR
# BASED ON:     OCR script by Salem (

TITLE=ScreenOCR # set yad variables

# - tesseract won't work if LC_ALL is unset so we set it here
# - you might want to delete or modify this line if you 
#   have a different locale:

export LC_ALL=en_US.UTF-8

# language selection dialog
LANG=$(yad \
    --width 300 --entry --title "$TITLE" \
    --image=$ICON \
    --window-icon=$ICON \
    --button="ok:0" --button="cancel:1" \
    --text "Select language:" \
    --entry-text \
    "eng" "ita" "deu")

# - You can modify the list of available languages by editing the line above
# - Make sure to use the same ISO codes tesseract does (man tesseract for details)
# - Languages will of course only work if you have installed their respective
#   language packs (

RET=$? # check return status

if [ "$RET" = 252 ] || [ "$RET" = 1 ]  # WM-Close or "cancel"

echo "Language set to $LANG"

SCR_IMG=`mktemp` # create tempfile
trap "rm $SCR_IMG*" EXIT # make sure tempfiles get deleted afterwards

scrot -s $SCR_IMG.png #take screenshot of area
mogrify -modulate 100,0 -resize 400% $SCR_IMG.png # postprocess to prepare for OCR
tesseract -l $LANG $SCR_IMG.png $SCR_IMG # OCR in given language
cat $SCR_IMG.txt | xsel -bi # pass to clipboard

Aside from the dependencies listed above you will need to install the Zenity fork YAD from the webupd8 PPA to make the script work.

share|improve this answer
works greate in terminal! thank you! I want to screencopy codetext from tutorials for testing. How to use scrot to clipboard? – Erling Apr 12 '13 at 15:07
what happens with the tempfiles? – Erling Apr 12 '13 at 15:18
The tempfiles stay there until you reboot your computer. If that is a problem to you, you can just delete them at the end (rm $SCR_IMG.png $SCR_IMG.txt). – Salem Apr 12 '13 at 15:27
scrot by itself can't use the clipboard. But there are tools like xclip or xsel that may do what you need if it is to copy/paste text. – Salem Apr 12 '13 at 15:28
Adding to Salem's answer: If you're running KDE then you can call another script to automatically send the text you've generated to the clipboard, ready to paste. You'll find a suitable script here. Follow the instructions on that page to install that script. Then all you need to is add | clipboard to the end of the final line of Salem's script. – Chris Jun 17 '13 at 18:59

I just done a blogging about how to use screenshot in modern day. Even though i target Chinese but the screen cast and code is in english. OCR is merely one of the feature.

Feature for my OCR:

  • Open in konsole+vimx OR gedit to further edit.

  • For vimx+english, enable spelling checking.

  • Support dynamic language selection without hard code.

  • Progress dialog when converting and tesseracting which is slow.

Function code:

function ocr () {
    atom="$(tesseract --list-langs 2>&1)"; atom=(`echo "${atom#*:}"`); atom=(`echo "$(printf 'FALSE\n%s\n' "${atom[@]}")"`); atom[0]='True'
    ans=(`yad --center --height=200 --width=300 --separator='|' --on-top --list --title '' --text='Select Languages:' --radiolist --column '✓' --column 'Languages' "${atom[@]}" 2>/dev/null`) && ans="$(echo "${ans:5:-1}")" &&  convert "$tmpj[x2000]" -unsharp 15.6x7.8+2.69+0 "$tmpocr_p" | yad --on-top --title '' --text='Converting ...' --progress --pulsate --auto-close 2>/dev/null && tesseract "$tmpocr_p" "$tmpocr" -l "$ans" 2>>/tmp/tesseract.log | yad --percentage=50 --on-top --title '' --text='Tesseracting ...' --progress --pulsate --auto-close 2>/dev/null && if [[ "$ans" == 'eng' ]]; then konsole -e "vimx -c 'setlocal spell spelllang=en_us' -n $tmpocr.txt" 2>/dev/null; else gedit "$tmpocr.txt"; fi
    rm "$tmpocr_p"

Caller code:

for cmd in "mktemp" "convert" "tesseract" "gedit" "konsole" "vimx" "yad"; do 
    command -v $cmd >/dev/null 2>&1 || {  LANG=POSIX; xmessage "Require $cmd but it's not installed.  Aborting." >&2; exit 1; }; :;
tmpj="$(mktemp /tmp/`date +"%s_%Y-%m-%d"`_XXXXXXXXXX.png)"
tmpocr="$(mktemp -u /tmp/`date +"%s_%Y-%m-%d"`_ocr_XXXXX)"
gnome-screenshot -a -f "$tmpj" 2>&1 >/dev/null | ts >>/tmp/gnome_area_PrtSc_error.log
ocr $tmpj $tmpocr $tmpocr_p &

Combine this 2 code in single shell script to run.

Screenshot 1: enter image description here

Screenshot 2: enter image description here

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.