Today, we are going to learn How can you automate your Computer Mouse and Keyboard using Python. In this article, we will be using a Python module PyAutoGUI that automates our mouse and keyboard. The PyAutoGUI library provides cross-platform support for managing mouse and keyboard It lets your Python scripts control the mouse and keyboard to automate interactions with other applications to enable automation of tasks. These scripts can control other applications by sending them virtual keystrokes and mouse clicks, just as if you were sitting at your computer and interacting with the applications yourself. The pyautogui library works on Windows, macOS, and Linux, and runs on Python 2 and 3. A tool like this has many applications, a few of which include taking screenshots, automating GUI testing (like Selenium), automating tasks that can only be done with a GUI, etc.
The API is designed to be as simple as possible with sensible defaults. But before we jump ahead, please note that there are a few prerequisites required. You should have a basic understanding of Python syntax. Other than that, We will be only going through the basics.
Also, you can find the full documentation at http://pyautogui.readthedocs.org/.
The pyautogui module can send virtual keypresses and mouse clicks to Windows, OS X, and Linux. The installation process for PyAutoGUI is fairly simple for all Operating Systems. However Depending on which operating system you%u2019re using, you may have to install some other modules before you can install PyAutoGUI.
On Windows, there are no other modules to install. Simply, Run the following Command In your Windows Terminal (cmd) and You should be able to get started with pyautogui.
On OS X, You need to Install some Dependencies Such as pyobjc-core and pyobjc along with pyautogui. To do so just run the following commands in your Terminal.
On Linux, You need to install certain packages such as python3-xlib, scrot (Scrot is a screenshot program that pyautogui uses.), python3-tk, and python3-dev along with pyautogui.
Let us start with some of the basic functions of pyautogui.
So we know the resolution of our display. Now we can move the mouse anywhere on the display within the limits of resolution of the display. In this section, we are going to cover pyautogui functions for mouse manipulation, which includes both moving the position of the cursor as well as clicking buttons automatically through code. We can move the mouse using the moveTo() method of pyautogui.
The example below of moving mouse to (100,100) position in 2 seconds.
We have passed three arguments to the moveTo() method. The first two arguments are compulsory and show the position of destination but the third argument is not compulsory. If we do not pass the third argument, then the mouse moves to the destination within no time.
We can position our mouse relative to the current position using the moveRel() Function as shown below
The above Line will move the cursor 100 points to the right and 100 points down in 2 seconds, with respect to the current cursor position.
Clicking, Scrolling, and Dragging Mouse
pyautogui provides some methods to control mouse actions. We can click, scroll and drag mouse using different methods.
The below example will move the position of the cursor and will perform a click action.
We can also execute specific click functions as follows.
The mouse scroll wheel can be simulated by calling the scroll() function and passing an integer number of clicks to scroll. The amount of scrolling in a click varies between platforms. Optionally, integers can be passed for the x and y keyword arguments to move the mouse cursor before performing the scroll. For example
Here, On OS X and Linux platforms, PyAutoGUI can also perform horizontal scrolling by calling the hscroll() function. For example
Before we move to the functions, it is important that we know which keys can be pressed through code in pyautogui, as well as their exact naming convention. To do so, run the following Comand in your Python Interpreter.
Key presses go to wherever the keyboard cursor is at function-calling time. The Typewrite function is used to write some text in a textbox field. for example
You can also use this Function to send a sequence of keyboard keys. To Implement this we will have to pass a List of keys sequence as an argument to the typewrite() Function. See the Example Below.
In the above example, A List of Keys will be entered in the textbox and will be submitted through the enter command.
Keyboard hotkeys like Ctrl-S or Ctrl-Shift-1 can be done by passing a list of key names to hotkey() function. We will see an example of how hotkey function works
Message Box Functions()
If you would like to take a screenshot of the screen at any instance, then you can use pyautogui inbuilt screenshot() function. pyautogui uses Pillow/Pil to work on Image related data but on Linux, you must run sudo apt-get install scrot to use the screenshot features. Let's see how we can implement that using pyautogui.
In the above example, we have stored an Instance of the screenshot in the variable named screenshot.
If you want to store the Screenshot directly to your system then you can specify the name of the png file in and pass it as an argument to the screenshot function directly.
You are now ready to automate your daily tasks through python pyautogui.