Sikuli | Tutorial | Download | Python GUI Automation Script

Contents

Sikuli | Tutorial | Download | Python GUI Automation Script

Sikuli | Tutorial | Download | Python GUI Automation Script-PURPOSE:

The purpose of this “Sikuli Tutorial” [ Sikuli | Tutorial | Download | Python GUI Automation Script ] is to provide a high level understanding of automation tool “SikuliX(Sikuli)”. Apart from basic understanding of “Sikuli” tool, we will be covering as much as “Sample Example Codes pertaining to Sikuli” which will certainly help reader to write robust code in real time Sikuli automation framework.

Sikuli | Tutorial | Download | Python GUI Automation Script- WHAT IS SIKULI?

Sikuli is an automation tool to automate anything which is visible on user’s screen and hence it is called as “WYSIWYS:What You See Is What You Script” tool. Sikuli recognizes objects on the basis of screenshot/images and snapshots.It uses OpenCV to recognize images.Automation Tester saves the screenshots of images for every object he needs to simulate a use case and when S(he) runs the script, Sikuli understands the definition of image and performs actions , mentioned in the script. It works well on all the OS i.e Windows, Mac or Linux/Unix. It is very useful tool when GUI definition i.e. “Xpath or Search/Filter Properties of the GUI Controls or object” is not accessible.

Sikuli | Tutorial | Download | Python GUI Automation Script- PREREQUISITE:

  • Valid Java- Please download Latest JAVA and install it to your system-Oracle/Sun Java version
  • Recheck Java Installation from command prompt. Type the below command at command prompt(CMD):
    • java -version
  • Out put should be similar to :
    • java version “1.8.0_91”
    • Java(TM) SE Runtime Environment (build 1.8.0_91-b15)
    • Java HotSpot(TM) 64-Bit Server VM (build 25.91-b15, mixed mode)

Sikuli | Tutorial | Download | Python GUI Automation Script- Sikuli SUPPORTED OS:

  • Windows-All Flavors- XP and Later till Win10(32 and 64 Bit)
  • Mac OSx-All Flavors- 10.6 and Later till 10.11(64 Bit)
  • Linux/Unix-All Flavors-Ubuntu (32 and 64 Bit)

Sikuli | Tutorial | Download | Python GUI Automation Script- Sikuli SUPPORTED SCRIPTING LANGUAGES:

  • Python-2.7 in the form of Jython.
  • Ruby 1.9 and 2.0 in the form of JRuby.
  • JavaScript in the form of Java Scripting Engine.
  • All the Java aware programming/scripting language (Jython, JRuby, Scala, Clojure, …)

Recommendation: Best and Easiest Language To Learn For Software Tester/Testing and Automation

SIKULI | TUTORIAL-HOW TO INSTALL SIKULI IN WINDOWS OS?

Sikuli Sikuli X Tutotials-How to Download SikuliX

SikuliX Download Page

  • Download the jar file to local folder i.e. C:\Sikuli.
  • Double click on Jar file and it will show the below Sikuli Setup window:

Sikuli Sikuli X Tutorials

SikuliX Setup Window

  • Check all the options as per your need.

Option 1 – It will setup SikuliX.Jar only which will invoke the default SikuliX IDE though you need to select of your preferred scripting language i.e. Python or Ruby. Apart from installing the Sikuli IDE, user can also run the script from command(CMD) prompt(line).

— Option 2- It will setup SikuliXAPI.Jar, which helps to write own custom code in JAVA or Java customized scripting language like JScript, JRuby,Jython, Scala,Clojure etc.The IDE to develop SikuliX scripts may be Eclipse or NetBeans or Pycharm.

— Option 3- It will setup “Tesseract-OCR”. User can use this option if user needs to perform any OCR related action like “Fetch or Get Text from Image”.

Sikuli | Tutorial | Download | Python GUI Automation ScriptSikuliX Setup Options

  • Once setup is completed , user can invoke the SikuliX IDE by clicking “runsikulix.cmd”.

 

SIKULI SIKULI X TUTORIALS-SHORTCUT USED IN SIKULI:

  • ALT+F-OPEN FILE MENU
  • ALT+E-OPEN EDIT MENU
  • ALT+R-OPEN RUN MENU
  • ALT+V-OPEN VIEW MENU
  • ALT+T-OPEN TOOLS MENU
  • ALT+H-OPEN HELP MENU
  • CTRL+A-OPEN ABOUT SIKULIX WINDOW
  • CTRL+N-OPEN NEW CODING WINDOW
  • CTRL+S-SAVE THE SCRIPT WRITTEN IN SIKULI IDE
  • CTRL+SHIFT+S-SAVE AS-DUPLICATE THE SCRIPT TO OTHER FILE
  • CTRL+SHIFT+E-EXPORT PACKED SOURCE
  • CTRL+W-CLOSE TAB IF MULTIPLE CODE WINDOW TABS ARE OPENED
  • CTRL+P-SET THE PREFERENCES
  • CTRL+SHIFT+2-TAKE THE SCREENSHOT

SIKULI | TUTORIAL-IS THERE ANY SIKULI BOOK AVAILABLE IN THE MARKET?

best books for sikuli test automation

Instant Sikuli Test Automation

As of now there is only one book available i.e. Instant Sikuli Test Automation in the market and seems the best book for learning Sikuli in the absence of any other book. As the name suggests it is really an instant book which focus on tool functionality and use of commands available in sikuli IDE. It is an easy to follow book which guides an Automation Tester to automate GUI based application. This book is a succinct guide for the QA professional who are newbie in the world of sikuli. Over this books is good for learning tool “SIKULI” though reader has to explore Python, Jython , Java, Java Script , JRuby to switch to advance level. Apart from learning these programming /scripting language user has to decide which language suite user’s need and hence best and easiest to go further. User has to learn Functions, Classes and other OOPs concept to define a robust automation framework and reusable component.

Other Recommended Books: Automation Testing , Manual Testing , Penetration Ethical Hacking Web Security Testing

SIKULI SIKULI X TUTORIALS-SIKULI BEST PRACTICES:

Automation Tester can adopt these below practices while writing and running the sikuli code though these are not specific to sikuli , it can be implemented to any generic automation project:

  • sikuli tutorial-sikuli best practices

    Sikuli Best Practices

    SIKULI BEST PRACTICES- SUGGESTION#1-Don’t rely on default captured images by sikuli IDE. Use your own tool like snippet or any other best tool to capture the image and save those images to automation framework’s image folder. While using the different sikuli commands like click, find, hover etc.,just pass the image path in parenthesis e.g. click(“F:\SikuliAutomation\Images\Home\HomeLogo.png”).

  • SIKULI BEST PRACTICES- SUGGESTION#2-Wait for Element(Web or Windows) to be fully appeared on the page or frame so rather than putting time based wait statement , Tester should pass object based wait statement.
  • SIKULI BEST PRACTICES- SUGGESTION#3-wait(30) is not a good practice to wait for next statement in sikuli test automation.
  • SIKULI BEST PRACTICES- SUGGESTION#4-using wait(“Next.png”,30) is good strategy to use in sikuli test automation.
  • SIKULI BEST PRACTICES- SUGGESTION#5-using waitVanish(“Next.png”)  is good strategy to use in sikuli test automation.
  • SIKULI BEST PRACTICES- SUGGESTION#6-Use proper naming convention while saving images to framework image repository.e.g. “BtnSubmit.png”
  • SIKULI BEST PRACTICES- SUGGESTION#7-Don’t feel shy to use Python code where ever needed.
  • SIKULI BEST PRACTICES- SUGGESTION#8-Always define reusable functions to perform redundant job.
  • SIKULI BEST PRACTICES- SUGGESTION#9-Try to use same kind of strategy which tester use in other automation frame. for ex.,define a Class for every page and embed methods to those classes. The benefit of this approach is, automation tester can define a POM(Page Object Model) kind of framework using Sikuli.
  • SIKULI BEST PRACTICES- SUGGESTION#10-Redefine  “Region”, or “Target Offset” in order to avoid any further confusion.
  • SIKULI BEST PRACTICES- SUGGESTION#11-Use the Log file to Test the Validation or Assertion.
  • SIKULI BEST PRACTICES- SUGGESTION#12-Use the HotKeys rather than IDE tools. for ex. “CTRL+SHIFT+2” can be used to capture the screenshot for image displayed on run-time(dynamic) e.g. capturing menu items etc.
  • SIKULI BEST PRACTICES- SUGGESTION#13-Code should be check-in after every update if it is being shared among the automation tester team members.
  • SIKULI BEST PRACTICES- SUGGESTION#14- Desktop should be clean and there should be dedicated system for recognizing the image and running the code.
  •  SIKULI BEST PRACTICES- SUGGESTION#15-Code, Modules, Functions, Classes and Methods should have proper comments to be easily debug the code in future by other members or by owner itself.

Many other best practices can be adopted as per size and nature of the project.Initially these practices look just wastage of time though over the period of time it become hard to manage the code and these practices really help us in those situations.

SIKULI CODE EXAMPLE(PYTHON):

SIKULI TUTORIAL EXAMPLE-Script to Open Calculator without using App.open or openApp function

#Script to Open Calculator without using App.open or openApp

cmd=r'"calc.exe"'

run(cmd)

wait(5)

type(Key.F4, KeyModifier.ALT)

SIKULI TUTORIAL EXAMPLE-Script to Open Calculator using App.open function

#Script to Open Calculator using App.open function

App.open("calc.exe")

wait(5)
type(Key.F4, KeyModifier.ALT)

SIKULI TUTORIAL EXAMPLE-Script to Open Calculator using openApp function

#Script to Open Calculator without using openApp function

openApp("calc.exe")

wait(5)
type(Key.F4, KeyModifier.ALT)

SIKULI TUTORIAL EXAMPLE-Script to Open google in Chrome browser and Search for a string “Testing”

Sikuli Script to Open google in Chrome browser and Search for a string “Testing”

Sikuli Script to Open google in Chrome browser and Search for a string “Testing”

SIKULI TUTORIAL EXAMPLE-Script to Select(check) all the unchecked check box or Unselect(Uncheck) all the Checked check box

In order to achieve the above goal , we first need to create a webform with multiple check boxes. Please paste the below code in any of the text editor(notepad,wordpad, textpad or notepad++ etc.) and save with file name “Test.Html”.


Open the file(WebPage) in any of the available browser(IE,Chrome,FF etc.)

Sikuli Script Example to Test CheckBoxes

Sikuli Script Example to Test CheckBoxes

Here is the Sikuli Code to Check or Uncheck check box:

#Script to Select(check) all the unchecked check box or Unselect(Uncheck) all the Checked check box
Tot_Checkbox=findAll(" ")
while Tot_Checkbox.hasNext():
    check=Tot_Checkbox.next()
    click(check)

SIKULI TUTORIAL EXAMPLE-How to run sikuli X script from command line(prompt)

In this section we will be covering Sikuli Tutorial Example to show “How to run sikuli X script from command line(prompt)”. It is super easy to run Sikuli X script from command in Windows OS. You can follow the below steps to run Sikuli X script from Command prompt. SIKULI SIKULI X TUTORIALS

  1. Locate the directory where SikuliX is installed.
  2. Check if “runsikulix.cmd” Windows Command Script exists in the folder(directory).
  3. Open the command prompt.
  4. Type PATH-TO-SIKULI\runsikulix -r PATH-TO-SIKULI X Script.sikuli  e.g. C:\>”F:\sikuli\runsikulix” -r “F:\RF\fetch picture.sikuli”
  5. Press Enter Key.
  6. Below is the snap shot of above mentioned steps:
SIKULI SIKULI X TUTORIALS EXAMPLE-How to run sikuli X script from command prompt

SIKULI SIKULI X TUTORIALS EXAMPLE-How to run sikuli X script from command line( prompt)

SIKULI TUTORIAL EXAMPLE-How to get coordinates of particular location and click on those coordinates with the help of Sikuli Script

There are the situations when Automation Tester doesn’t want to click on picture element , rather tester wants to click on the coordinates of that location.Here are the steps to achieve both the goals:

  1. Open new script page in the Sikuli IDE.
  2. Type the command print Env.getMouseLocation() in the IDE Editor.
  3. Save the Script.
  4. Navigate mouse to the desired location e.g. Windows Start Button.
  5. Press Ctrl+R , Which will run the script.
  6. In the message window of Sikuli IDE , it will show the coordinates of the location e.g. L(25,742)@S(0)[0,0 1366×768]
  7. Now Tester know the coordinates of the location and by writing the below command in the IDE , Tester can click on that location:
    • click(Location(25,742))
#Script to get coordinates of particular location and click on those coordinates with the help of Sikuli Script
#click("1488881260535.png")
wait(3)
print Env.getMouseLocation()
click(Location(24,747))

SIKULI TUTORIAL EXAMPLE-How to capture and save screen shot(s) of current screen or app with the help of Sikuli Script(Python)

There are the situations when Automation Tester wants to capture and save screen shot(s) of current screen or app.Here are the steps and Python script to capture and save screen shot of current screen :

  1. Open new script page in the Sikuli IDE.
  2. Type the command mentioned in the bottom in the IDE Editor.
  3. Save the Script.
  4. Press Ctrl+R , Which will run the script.
#Script to capture and save screen shot(s) of current screen or app with the help of sikuli script(Python)
import shutil
screen_shot=capture(SCREEN)
shutil.move(screen_shot, '/TestResult/Images/AppImage.png')

SIKULI TUTORIAL EXAMPLE-How to fix issue “[error] SyntaxError ( “no viable alternative at input ‘1490771802872.’”, )”

Sometimes while running the sikuliX script ,we get error like “[error] SyntaxError ( “no viable alternative at input ‘1490771802872.’”, )”.Here are the steps and Python script to fix “[error] SyntaxError ( “no viable alternative at input ‘1490771802872.’”, )”:

  1. Navigate to Sikuli folder where all the installed files are located.
  2. Double click on “runsikulix.cmd”
  3. Open your script page in the Sikuli IDE.
  4. Look for following syntactical error:
    1. Check if any image you would like work on , is enclosed in double quote. for ex.click(“”1490771802872.png””). Please remove double quotes.
    2. Check if any image you would like work on , does not have end(close) parenthesis. for ex.click(“1490771802872.png”. Please put parenthesis at end of the command/script.
  5. Save the Script.
  6. Press Ctrl+R , Which will run the script with out throwing any error.

Before Fix

no viable alternative at input

 

After Fix
Issue Solved-no viable alternative at input

SIKULI TUTORIAL EXAMPLE-How to run sikuli X script from command line(prompt) Batch File(.cmd extension)

There are situations when Sikuli Automation Tester wants to run sikuli script in batch file which will run the sikuli script in just one click rather then running the script in Sikuli IDE or in Command line.In this section we will be covering Sikuli Tutorial Example to show “How to run sikuli X script from command line(prompt) Batch File(.cmd extension)”. It is super easy to run Sikuli X script from command in Windows OS. You can follow the below steps to run Sikuli X script from Command prompt. SIKULI SIKULI X TUTORIALS

  1. Locate the directory where SikuliX is installed.
  2. Check if “runsikulix.cmd” Windows Command Script exists in the folder(directory).
  3. Open notepad.
  4. Type PATH-TO-SIKULI\runsikulix -r PATH-TO-SIKULI X Script.sikuli  e.g. C:\>”F:\sikuli\runsikulix” -r “F:\RF\fetch picture.sikuli” or C:\>”F:\sikuli\runsikulix” -r “F:\RF\fetch picture.sikuli”
  5. Save note pad with file extension .cmd e.g. “validation.cmd”
  6. Double click over “validation.cmd” file.
  7. It will run the .sikuli or .skl script file successfully.
  8. Below is the snap shot of above mentioned steps:

    SIKULI SIKULI X TUTORIALS EXAMPLE-Run Sikuli Script with out Sikuli IDE

    SIKULI SIKULI X TUTORIALS EXAMPLE-Run Sikuli Script with out Sikuli IDE

SIKULI TUTORIAL EXAMPLE-How to pick image from desired location in Sikuli Script

There are situations when Sikuli Automation Tester wants to pull image(s) to be  used in sikuli script from the desired location rather than default image location set by Sikuli. In this section we will be covering Sikuli Tutorial Example to show “How to pick image from desired location in Sikuli Script“.There are many ways to achieve this goal .

Method #1: Save the image to desired location and populate the absolute image path to be used in Sikuli Script.

  1. Let say our project folder is “C:\SikuliScripts\UIValidation”
  2. Create Another Folder under “C:\SikuliScripts\UIValidation” with name Images i.e.””C:\SikuliScripts\UIValidation\Images”
  3. Save All the Images in this folder and this folder will work as an Image(Object) Repository.
  4. Tester can perform any action after giving absolute path. For Ex. click(“C:\\SikuliScripts\\UIValidation\\Images\\Calculator.PNG”)

Method#2: Add Image Path

  • Let say our project folder is “C:\SikuliScripts\UIValidation”
  • Create Another Folder under “C:\SikuliScripts\UIValidation” with name Images i.e.”C:\SikuliScripts\UIValidation\Images”
  • Save All the Images in this folder and this folder will work as an Image(Object) Repository.
  • Now Add this path addImagePath(“C:\\SikuliScripts\\UIValidation\\Images”)
    click(“..\\Calculator.png”)
  • Tester can perform any action after giving Relative path. For Ex. click(“..\\Calculator.PNG”)

SIKULI TUTORIAL EXAMPLE-How to pull data from Text file in Sikuli Script

There are situations when Sikuli Automation Tester wants to pull certain data to be used in sikuli script from the text file. In this section we will be covering Sikuli Tutorial Example to show “How to pull data from Text file in Sikuli Script“. Suppose the scenario is, there is a log file which contains “Current Network Bandwidth” info at different time span and Tester needs to pull all the counter data from the log file. Here is the sample file which contains raw data:

Counter=1 DateTime-2018-01-19 15:11:51<PARAM NAME=”Counter00004.Path” VALUE=”\Network Interface(vmxnet3 Ethernet Adapter)\Current Bandwidth”/> <PARAM NAME=”Counter00004.Data” VALUE=”10000000000″/> <PARAM NAME=”Counter00005.Path” VALUE=”\Processor Information(_Total)\% Processor Time”/>
Counter=2 DateTime-2018-01-19 15:21:51<PARAM NAME=”Counter00004.Path” VALUE=”\Network Interface(vmxnet3 Ethernet Adapter)\Current Bandwidth”/> <PARAM NAME=”Counter00004.Data” VALUE=”10000000663″/> <PARAM NAME=”Counter00005.Path” VALUE=”\Processor Information(_Total)\% Processor Time”/>

From the above sample data we need to pull the DateTime and Bandwidth data for each occurrence. Here are the steps:

  1. Save the data in a file named C:\Log.txt.
  2. Invoke SikuliXIDE and Open a new Script Window.
  3. Import with statement in sikuli script.
  4. Store all the data variable to be used in a List.
  5. Read the file line by line and search for data variable in each line.
  6. Print the variable data.
  7. Here is the script.

from __future__ import with_statement
with open("C:\\Log.txt") as f:
    i=0
    for line in f:
        i=i+1
        mylist=['DateTime','\"Counter00004.Data\"']
        count=len(mylist)
        for T in range(count):
            if mylist[T] in line:
                print line,i

 

OutPut:
 
DateTime-2018-01-19 15:11:51
2
<PARAM NAME=”Counter00004.Data” VALUE=”10000000000″/>
4
DateTime-2018-01-19 15:21:51
8
<PARAM NAME=”Counter00004.Data” VALUE=”10000000663″/>
10

SIKULI TUTORIAL EXAMPLE-How to pull data from Excel i.e. XLS/XLSX file in Sikuli Script

There are situations when Sikuli Automation Tester wants to pull certain data to be used in sikuli script from the Excel i.e. XLS/XLSX file. In this section we will be covering Sikuli Tutorial Example to show “How to pull data from Excel i.e. XLS/XLSX file in Sikuli Script“. Here is the sample data from an excel file:

Test           Hi

best            2

cast

Here are the steps to fetch data from excel sheet in sikuli:

  1. Save the excel sheet in the same folder where Sikuli is installed. In my case it is “F:\Sikuli\”
  2. Create a new Folder with Name “Lib”.
  3. Download the xlrd library from “https://pypi.python.org/pypi/xlrd#downloads” .
  4. Unzip it with the help of 7Zip and save it to Lib folder.
  5. Rename the XLRD folder to XLRD2.
  6. Invoke SikuliXIDE and Open a new Script Window.
  7. Read Number of Cols and Rows from the file.
  8. Use the For loop with in the range of Rows and Cols.
  9. Print the Cell Value for the current Row and Col.
  10. Here is the Python script.
import xlrd2
from xlrd2 import open_workbook
book = xlrd2.open_workbook("test.xls")
sh = book.sheet_by_index(0)
for rx in range(sh.nrows):
    for cx in range(sh.ncols):
#        cell_obj = sh.cell(rx, cx)
#        print cell_obj.value
        print sh.cell(rx,cx).value,
    print
OutPut:
best 2.0
cast

6 Comments

  1. Allen May 12, 2017
  2. Mohit November 14, 2017
  3. Anup Kumar November 17, 2017
  4. Odmbie November 18, 2017
  5. Rick Johnson November 20, 2017
  6. Swati Mancha November 23, 2017

Leave a Reply

error: Content is protected !!