Python pyautogui - An Introduction














































Python pyautogui - An Introduction



Introduction

Today, we are going to learn How can you automate your Computer Mouse and Keyboard using Python. In this article, we will be using a Python module PyAutoGUI that automates our mouse and keyboard. The PyAutoGUI library provides cross-platform support for managing mouse and keyboard It lets your Python scripts control the mouse and keyboard to automate interactions with other applications to enable automation of tasks. These scripts can control other applications by sending them virtual keystrokes and mouse clicks, just as if you were sitting at your computer and interacting with the applications yourself. The pyautogui library works on Windows, macOS, and Linux, and runs on Python 2 and 3. A tool like this has many applications, a few of which include taking screenshots, automating GUI testing (like Selenium), automating tasks that can only be done with a GUI, etc.

The API is designed to be as simple as possible with sensible defaults. But before we jump ahead, please note that there are a few prerequisites required. You should have a basic understanding of Python syntax. Other than that, We will be only going through the basics.

Also, you can find the full documentation at http://pyautogui.readthedocs.org/.


Installation

The pyautogui module can send virtual keypresses and mouse clicks to Windows, OS X, and Linux. The installation process for PyAutoGUI is fairly simple for all Operating Systems. However Depending on which operating system you%u2019re using, you may have to install some other modules before you can install PyAutoGUI.

Windows

On Windows, there are no other modules to install. Simply, Run the following Command In your Windows Terminal (cmd) and You should be able to get started with pyautogui.

pip install pyautogui 

Os X 

On OS X, You need to Install some Dependencies Such as pyobjc-core and pyobjc along with pyautogui. To do so just run the following commands in your Terminal.

sudo pip3 install pyobjc

sudo pip3 install pyobjc-core

sudo pip3 install pyautogui

Linux 

On Linux, You need to install certain packages such as python3-xlib, scrot (Scrot is a screenshot program that pyautogui uses.), python3-tk, and python3-dev along with pyautogui.

sudo pip3 install python3-xlib

sudo apt-get install scrot

sudo apt-get install python3-tk

sudo apt-get install python3-dev.

sudo pip3 install pyautogui 

Getting Started

Let us start with some of the basic functions of pyautogui.

Basic Functions()


>>> import pyautogui

>>> pyautogui.position()         # current mouse x and y Position 

(968, 56) 

>>> pyautogui.size()                # current screen resolution width and height 

(1920, 1080) 

>>> pyautogui.onScreen(x, y) # True if x & y are within the screen. 

True 

Mouse Functions()

So we know the resolution of our display. Now we can move the mouse anywhere on the display within the limits of resolution of the display. In this section, we are going to cover pyautogui functions for mouse manipulation, which includes both moving the position of the cursor as well as clicking buttons automatically through code.  We can move the mouse using the moveTo() method of pyautogui.  

The example below of moving mouse to (100,100) position in 2 seconds.


>>> Import pyautogui

>>> pyautogui.moveTo(x, y, duration=2


We have passed three arguments to the moveTo() method. The first two arguments are compulsory and show the position of destination but the third argument is not compulsory. If we do not pass the third argument, then the mouse moves to the destination within no time.

We can position our mouse relative to the current position using the moveRel() Function as shown below 


>>> import pyautogui

>>> pyautogui.moveRel(100,100,2)


The above Line will move the cursor 100 points to the right and 100 points down in 2 seconds, with respect to the current cursor position.


Clicking, Scrolling, and Dragging Mouse


pyautogui provides some methods to control mouse actions. We can click, scroll and drag mouse using different methods.

The below example will move the position of the cursor and will perform a click action.


>>> import pyautogui

>>> pyautogui.moveTo(300,301)

>>> pyautogui.click()


We can also execute specific click functions as follows.


>>> import pyautogui

>>> pyautogui.moveTo(300,301)

>>> pyautogui.doubleClick()

>>> pyautogui.rightClick()

>>> pyautogui.tripleClick()

>>> pyautogui.middleClick()


Mouse Scrolling


The mouse scroll wheel can be simulated by calling the scroll() function and passing an integer number of clicks to scroll. The amount of scrolling in a click varies between platforms. Optionally, integers can be passed for the x and y keyword arguments to move the mouse cursor before performing the scroll. For example

>>> pyautogui.scroll(10)   # scroll up 10 clicks

>>> pyautogui.scroll(-10)  # scroll down 10 clicks

>>> pyautogui.scroll(10, x=100, y=100)  # move mouse cursor to 100, 200, then scroll up 10 clicks

 

Here, On OS X and Linux platforms, PyAutoGUI can also perform horizontal scrolling by calling the hscroll() function. For example

>>> pyautogui.hscroll(10)   # scroll right 10 clicks

>>> pyautogui.hscroll(-10)   # scroll left 10 clicks

Keyboard Functions()

Before we move to the functions, it is important that we know which keys can be pressed through code in pyautogui, as well as their exact naming convention. To do so, run the following Comand in your Python Interpreter.


>>> print(pyautogui.KEYBOARD_KEYS)


Key presses go to wherever the keyboard cursor is at function-calling time. The Typewrite function is used to write some text in a textbox field. for example


>>> pyautogui.typewrite(Your_Text, interval = 2)


You can also use this Function to send a sequence of keyboard keys. To Implement this we will have to pass a List of keys sequence as an argument to the typewrite() Function. See the Example Below.


>>> pyautogui.typewrite([Key1,Key2,Key3])


In the above example, A List of Keys will be entered in the textbox and will be submitted through the enter command.


Keyboard hotkeys like Ctrl-S or Ctrl-Shift-1 can be done by passing a list of key names to hotkey() function. We will see an example of how hotkey function works


>>> pyautogui.hotkey(Command1,Command2)            

>>> pyautogui.hotkey(Command1,Command2)            

Message Box Functions()

If you need to pause the program until the user clicks OK on something, or want to display some information to the user, the message box functions have similar names that JavaScript has. Let us see this through an example.


>>> pyautogui.alert(Message)

>>> pyautogui.confirm(Message)

>>> pyautogui.prompt(Message)

Screenshot Functions()

If you would like to take a screenshot of the screen at any instance, then you can use pyautogui inbuilt screenshot() function. pyautogui uses Pillow/Pil to work on Image related data but on Linux, you must run sudo apt-get install scrot to use the screenshot features. Let's see how we can implement that using pyautogui.


>>> screenshot = pyautogui.screenshot()


In the above example, we have stored an Instance of the screenshot in the variable named screenshot.


If you want to store the Screenshot directly to your system then you can specify the name of the png file in and pass it as an argument to the screenshot function directly.


>>> pyautogui.screenshot(Your_Image_Name)


You are now ready to automate your daily tasks through python pyautogui.



Comments