Skip to content

Selenium

Introduction to Selenium

Selenium is a powerful tool for automating web browsers. It is widely used for automating web applications for testing purposes, but it can also be used for web scraping and automating routine web tasks. In this guide, we'll walk you through setting up Selenium with Python and performing basic web automation tasks. The documentation assumes you have basic Python knowledge.

1. Prerequisites

Before getting started with Selenium, make sure you have the following: - Basic knowledge of Python (working with variables, functions, loops, etc.) - Python installed on your computer (preferably Python 3) - Web browser (e.g., Chrome, Firefox)

Install Required Packages

Selenium can be installed easily via pip. Open your terminal or command prompt and type the following:

pip install selenium

Additionally, you will need a WebDriver to interact with a specific browser.

WebDriver Setup

  • Chrome: Download the ChromeDriver that matches your version of Chrome.
  • Firefox: Download the GeckoDriver for Firefox.

After downloading the WebDriver, add it to your system PATH or specify its path in your code.

2. Getting Started with Selenium

Importing Selenium

To use Selenium in your Python script, import it as follows:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
import time

Starting a Web Browser

To start a browser session, instantiate the WebDriver:

# Using Chrome
driver = webdriver.Chrome(executable_path='path/to/chromedriver')

# Using Firefox
# driver = webdriver.Firefox(executable_path='path/to/geckodriver')

# Open a website
driver.get("https://www.example.com")

Basic Operations

  1. Open a Webpage:
driver.get("https://www.google.com")
  1. Maximize the Browser Window:
driver.maximize_window()
  1. Interacting with Web Elements:

You can interact with elements on a web page, such as entering text into input fields, clicking buttons, etc.

Finding elements:

search_box = driver.find_element(By.NAME, "q")  # Finding an element by name
search_box.send_keys("Selenium Python")        # Typing into the search box
search_box.send_keys(Keys.RETURN)              # Pressing Enter

Clicking a button:

button = driver.find_element(By.ID, "buttonID")
button.click()
  1. Waiting for Elements to Load: Sometimes, elements on a webpage may not load immediately. You can make Selenium wait for an element to appear before interacting with it.
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

# Wait for an element to be present (max wait time: 10 seconds)
element = WebDriverWait(driver, 10).until(
    EC.presence_of_element_located((By.ID, "elementID"))
)
  1. Handling Delays:

If you want the script to wait for a specific amount of time:

time.sleep(5)  # Wait for 5 seconds

3. Common Selenium Commands

Here are some of the most commonly used commands in Selenium:

  • Find Element by ID:

python element = driver.find_element(By.ID, "element_id")

  • Find Element by Name:

python element = driver.find_element(By.NAME, "element_name")

  • Find Element by Class Name:

python element = driver.find_element(By.CLASS_NAME, "class_name")

  • Find Element by CSS Selector:

python element = driver.find_element(By.CSS_SELECTOR, ".class_name #id_name")

  • Find Element by XPATH:

python element = driver.find_element(By.XPATH, "//tag[@attribute='value']")

  • Click an Element:

python element.click()

  • Enter Text into a Field:

python element.send_keys("Your text here")

  • Clear Input Field:

python element.clear()

4. Navigating Through Pages

You can use the following commands to navigate through the web:

  • Go to a URL:

python driver.get("https://www.example.com")

  • Navigate Back:

python driver.back()

  • Navigate Forward:

python driver.forward()

  • Refresh the Page:

python driver.refresh()

5. Handling Alerts and Pop-ups

If a website has alerts or pop-ups, you can handle them using:

  • Accepting an Alert:

python alert = driver.switch_to.alert alert.accept()

  • Dismissing an Alert:

python alert = driver.switch_to.alert alert.dismiss()

6. Closing the Browser

Once you've completed your automation tasks, it’s important to close the browser:

driver.quit()

7. Complete Example: Google Search Automation

Here's a full example that demonstrates opening Google, searching for something, and closing the browser.

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
import time

# Start the WebDriver (Chrome in this case)
driver = webdriver.Chrome(executable_path="path/to/chromedriver")

# Open Google
driver.get("https://www.google.com")

# Find the search box
search_box = driver.find_element(By.NAME, "q")

# Type in search query
search_box.send_keys("Selenium Python")

# Press Enter
search_box.send_keys(Keys.RETURN)

# Wait for 5 seconds to see the results
time.sleep(5)

# Close the browser
driver.quit()

8. Best Practices

  • Always make sure the WebDriver version matches your browser version.
  • Use waits (WebDriverWait or time.sleep()) to ensure elements load correctly before interacting.
  • After interacting with a web element, verify that the action was successful (e.g., by checking the page content).

Additional Resources

This guide covers the basics of Selenium with Python. By practicing these steps, you can automate web browsers for a variety of purposes, such as testing and web scraping.