Some of my routine tasks involve interaction with web pages, often to fill in one or more web forms with data that can be provided programmatically. With my previous email provider, I used this to add new email aliases to my email account. Now, I need it to send documents to our work printers. To ease these tasks, I developed a very simple (and limited) Python-based web interaction scripting language. These scripts can be executed using my webinteract Python module implemented with the Python framework splinter. The splinter framework provides an interface for web application automation. Default, the webinteract module uses the splinter framework with the chrome driver. For this to work, both Chrome and ChromeDriver have to be installed.

The webinteract module has been developed to fit my needs. New features have been added when needed. It is not made to be complete or robust, but it is made to be easily extendable. New action (command) types can be easily added by adding methods to a class in the module (or extending that class). For more advanced use cases, it might be better to use the splinter framework directly. But for some types of automated web page interactions, the webinteract module might be an easy-to-use approach.

The webinteract module is available here:

https://www.pg12.org/dist/py/lib/webinteract/

The easiest way to use and install the module is to use pip (webinteract at PyPi):

pip install webinteract

Table of contents

In the following text, I will first introduce web interaction actions and how you write scripts for web interaction tasks. Next, I will show how to run such scripts from the command line. Then, I will document the currently available web interaction actions of the module. At the end, I have added a section with some comments and hints for maintaining and troubleshooting the software involved.

• Web interaction actions
• Use the module as a program
• The web interaction actions reference
• Comments and troubleshooting

Web interaction actions

My webinteract module interprets and executes web interaction scripts. The web interaction script commands are called web interaction actions, and files containing such scripts use the wia extension. This is an example of a wia script with web interaction actions:

setvals(url="https://a.web.page/")
setvals(account="an@email.address", pw="s3crEt p4s5w0rd")
visit(url)
fill(account, "id", "loginForm:id")
fill(pw, "id", "loginForm:password")
click("id", "loginForm:loginButton")
verify( \
  is_text_present("Login succeeded", wait_time=10), True, \
  "Login failed")
fill("A text", "id", "textForm:field1")
fill("Another text", "id", "textForm:field2")
click("id", "registerForm:addText")
verify( \
  is_text_present("Add text failed"), False, \
  "Add text failed")

The actions in the first two lines of the script set the values of three variables, url, account and pw. In line 3, the action opens the web page at the address given in the variable url. The actions in lines 4 and 5 fill in the two fields with the IDs "loginForm:id" and "loginForm:password" in the web form with values from the variables account and pw, respectively. The action in line 6 clicks a login button (with the ID "loginForm:loginButton"). In line 7 to 9, we perform a check: is the text "Login succeeded" present on the web page? Since it might take a while for a web page to completely load (after the action at line 6 pressed the button), we specify that the check should wait (at most) 10 seconds for this to become true. These three lines combine two web interaction actions: verify and is_text_present. The result of the is_text_present action is the first argument to the verify action. We also see that these three lines are actually combined to one line when interpreted. The \ at the end of a line removes the line break (and any extra spaces at the beginning of the next line) before the combined line is interpreted. We expect the result to be true; if not, the script fails and terminates here. The following actions in lines 10, 11, 12, 13, 14 and 15 fill in another web form with two text fields and check that the error message "Add text failed" is not present after the register button is pressed.

Use the module as a program

There are two approaches to using the web module as a program. This first approach can be used if you have downloaded the module file from my file repository. The second apprach can be used if you intalled the module with pip.

Installed module manually

If the example above is stored in a file webinteraction-example.wia it can be executed using the following command:

python3 webinteract.py webinteraction-example.wia

If the module is installed as the program webinteract at a location found by your shell, a simpler way to execute the script is possible:

webinteract webinteraction-example.wia

The first line of the webinteract.py module contains this magic string:

#!/usr/bin/env python3

The magic string says this is a Python program, and the system knows it should use Python 3 to execute the program. When this file is copied with the name webinteract to a location (path) where your shell looks for executables, the program can be executed from the command line independent of the current directory of your shell. You have to ensure that the program file is executable. If the directory ~/bin is one of the directories your shell is looking for executables, you can make webinteract.py such an executable with the following two steps:

cp webinteract.py ~/bin/webinteract
chmod +x ~/bin/webinteract

The first line copies the file to a location where your shell will find it and renames it to webinteract. The second line makes the file executable. The locations your shell looks for executables (programs) can changed by updating the environment variable PATH. It is out of the scope of this blog post to discuss how this is done.

If the webinteract program is installed at a location found by the shell, we can further simplify the executions of such scripts using the magic string trick again. Instead of using webinteract at the command line to execute a wia script, we can insert a magic string into the script itself. Insert the following magic string as the first line in your wia scripts:

#!/usr/bin/env webinteract

If you have made the script executable with the chmod command, you should be able to execute the script from the current directory directly:

./webinteraction-example.wia

The webinteract program (the webinteract Python module used as a program) provides a few command line arguments that can modify its behaviour (see the documentation of all command line arguments below). We can, for example, run the example script headless using this command:

webinteract --headless webinteraction-example.wia

Or, if we have applied the necessary magic to the script itself, this will also work:

./webinteraction-example.wia --headless

Installed module with pip

If you have installed the module with pip, you do not have to locate the webinteract.py file, copy it to a location where your shell looks for executables, and make it executable. You can run the module as a program using the -m command line argument to Python:

python3 -m webinteract

This means that when the module is installed with pip, you can use this magic string in your wia scripts:

#!/usr/bin/env python3 -m webinteract

And, of course, command line arguments will still work:

./webinteraction-example.wia --headless

In your shell, you can make webinteract and alias where Python executes the module as a program (check the documentation in your preferred shell):

alias webinteract='python3 -m webinteract'

Use namespace and the keychain with command line arguments

A more advanced usage of command line arguments to the webinteract program might involve your computer's keychain (system keyring) and a provided namespace where variables are assigned values. Let us start with this wia script example add-alias.wia. Remember to make the script executable, and if the moudle is installed with pip use the alternative magic string in the first line:

#!/usr/bin/env webinteract
visit("https://email.provider/controlpanel")
fill(account, "id", "loginForm:id")
fill(pw, "id", "loginForm:password")
click("id", "loginForm:loginButton")
verify( \
  is_text_present("Login succeeded", wait_time=10), True, \
  "Login failed")
fill(alias, "id", "textForm:alias")
click("id", "registerForm:addAlias")
verify( \
  is_text_present("Add alias failed"), False, \
  "Add alias failed")

In this example, we log in to a web page with an account and a password. If this succeeds, we add an alias and verify that the error message does not occur. In the example, three unset variables are used: account, pw, and alias. The pw variable is the password used to log in. This should not be stored in clear text anywhere. To avoid this, webinteract can fetch the password from a keychain. It uses the keyring module to implement it. By providing the account name and service name used when the password was registered in the keychain, webinteract will fetch the password from the keychain and store it in the variable pw. When executing the script, the two other unset variables can be given values using the namespace argument. If the log in password is stored under the account name my@email.address and the service name email.provider, an example of running the script could be:

./add-alias.wia \
    --pw-account my@email.address \
    --pw-service email.provider \
    --name-space '{"account": "my@email.address", "alias": "new-alias"}'

Since the account name used to access the password from the keychain is the same as the account variable in the namespace, we can simplify the usage of the script by removing the account variable from the namespace and use the pw_account variable instead:

./add-alias.wia \
    --pw-account my@email.address \
    --pw-service email.provider \
    --name-space '{"alias": "new-alias"}'

In the script, we then have to change line 3 to use the pw_account variable instead of the account variable:

fill(pw_account, "id", "loginForm:id")

Interactive usage

The webinteract module can also be used as an interactive program with a prompt where you can write your web interaction actions. For example, start the webinteract program with the command webinteract (no wia file given) and you will be given the following prompt:

wia> 

The web-browser will start with no page loaded. In this prompt, you can type in the web interaction actions and watch the effect of the actions in the open web-browser window:

wia> visit("https://a.web.page/")
wia> click_link("partial_href", "news")

You can also pass arguments to the program when used in interactive mode. For example, if log-in information is needed, you should grab it from the keychain:

webinteract --pw-account my@email.address --pw-service email.provider

Then you can use this information interactively in the web interaction actions:

wia> visit("https://a.web.page/")
wia> fill(pw_account, "id", "loginForm:id")
wia> fill(pw, "id", "loginForm:password")
wia> click("id", "loginForm:loginButton")

The interactive prompt is also useful when debugging your scripts. For example, I use it to see what find actions return or the value or text of web elements of a web page. If an action returns a value, the value is displayed in the interactive prompt. Some examples (including the values displayed in the prompt):

wia> setvals(select_elem = find("id", "selector"))
wia> element_get_value(select_elem)
'tag'
wia> element_get_text(select_elem)
'tag\nyear\nsearch'
wia> get_text("tag", "h1", 0)
'A web page'

Besides the web interaction actions, a few other commands are available when webinteract is used interactively. These commands are used to print and manage the action command history and to show help texts:

• history: List command history.
• hdrop: Drop last command from history.
• hclear: Clear command history completely.
• help: Print help text.
• doc: Print module documentation.

If you add an action after the help command, the help text for that specific action is provided (e.g., the command «help click» print the documentation for the click action). With the help command, the interactive prompt of webinteract is a nice way to access the documentation of the webinteract module and its web interaction actions.

Scrape web page content

Another usage of the webinteract module is to scrape a web page for content (data). In this example script, get_text.wia, we get the content of the third paragraph on the web page:

#!/usr/bin/env webinteract
visit("https://a.web.page/")
get_text("tag", "p", 3)

If we run this script without any arguments, the content is written to standard output (stdout). If we want to write it to a file, we can use the --output-file argument:

./get_text.wia --output-file thetext.txt

The output is then written to the file thetext.txt. In interactive mode, we can use the --output-file argument to save the results to a file:

webinteract --output-file thedata.txt
wia> visit("https://a.web.page/")
wia> get_text("tag", "p", 3)
'The text at paragraph three on the web page.'
wia> exit
cat thedata.txt
The text at paragraph three on the web page.

Command line arguments

Positional arguments:

• script
  • The web script file name (path), - for stdin.

Options:

• -h, --help
  • Show the help message and exit. The help message includes a description of the command line arguments.
• -n, --name-space NAME_SPACE
  • Add variables to namespace (JSON-format).
• -s, --pw-service PW_SERVICE
  • Password stored at this service name in the keychain. The program uses the keyring module to look for the service name in the keychain. The service name is also stored in the namespace with the variable name pw_service.
• -a, --pw-account PW_ACCOUNT
  • Password stored at this account name in the keychain. The program uses the keyring module to look for the account name in the keychain. The account name is also stored in the namespace with the variable name pw_account.
• -o, --output-file OUTPUT_FILE
  • Any output is written to this file (default stdout).
• --driver DRIVER
  • The web interaction driver name, where the default is chrome. For other options, see the splinter Browser and Drivers documentation.
• --headless
  • Run browser headless (invisible). Requires newer Chrome or Firefox browsers.
• --keep
  • Keep the browser running after the script has terminated.
• --prompt PROMPT
  • The text of the interactive prompt (more on interactive prompt below).
• --prompt-color PROMPT_COLOR
  • The text color of the interactive prompt.
• --prompt-output-color PROMPT_OUTPUT_COLOR
  • The text color of the interactive output.
• --prompt-alert-color PROMPT_ALERT_COLOR
  • The text color of the interactive alerts.

The web interaction actions reference

The webinteract module provides the following web interaction actions:

• wait(wait_time: int = 1)
  • wait_time: The amount of time to wait in seconds (the default is 1)

  Wait for the given number of seconds. Used if you need to wait for web elements to be loaded or when debugging web interaction action scripts. Some other actions, like is_present, can also wait for a while if the expected element is not present yet (they also have an optional argument wait_time). An example of waiting for four seconds:

  wait(4)

• setvals(**kw)
  • kw: Named arguments that set values in the namespace.

  The setvals action can give values to one or more variables used in other actions later in the script. The setvals action updates the script's namespace with the variables and their given values. The following example sets the values of the two variables url and email:

  setvals(url = "https://a.web.page/", email = "an@email.address")

  It is also possible to use web interaction actions that return a value with setvals. In the following example, we set the value of the variable tag to the value of an element with an id "tag":

  setvals(tag = get_value("id", "tag"))

• verify(va1: Any, va2: Any, errmsg: str = "No match", assert_it: bool = True) -> bool
  • val1: Value one.
  • val2: Value two.
  • errmsg: The error message given if the verification fails and the WebInteractionError is raised (default "No match").
  • assert_it: Should the action raise the WebInteractionError if the values don't match (default True).
  • return: True if the two values match, False otherwise. If the assert_it argument is True (default), the WebInteractionError exception is raised if the two values do not match, and if they match, nothing is returned (and nothing happens).

  The action verify checks that the two given value arguments match. If they don't match and the assert_it argument is True (the default), the WebInteractionError is raised (the wia-script terminates). If they don't match and the assert_it argument is False, the action returns False. Each value argument va1 and va2 is either a value or an action. If it is an action, the action is performed and the result of the action is the value compared with the other argument. A simple example where we compare a variable with at text string:

  verify(url, "https://a.web.page/")

  A WebInteractionError exception is raised if they don't match. Another example where we check if a web element with the id "afilter" is present:

  verify(is_present("id", "afilter"), True, "No filter")

  Again, a WebInteractionError exception is raised if is_present returns False. The exception is raised with the error message "No filter". If the assert_it argument is False, verify can be used in combination with other actions:

  setvals( \
  searchweb = verify( \
    "web", get("id", "searchtxt"), assert_it = False))

• visit(url: str)
  • url: The URL to be visited.

  The action visit opens a web page (URL). The actions that follow will interact with this web page. This example opens the web page "https://a.web.page/":

  visit("https://a.web.page/")

• find(stype: SelectorType, sval: str) -> ElementList
  • stype: The selector type.
  • sval: The value of the selector type.
  • return: A list of the web elements matching the selector stype with the value sval.

  The action find returns web elements based on a selector type and the value of such a selector. This example returns a list of web elements with id set to "filter" (often a list with a single element):

  find("id", "filter")

  Another example using an XPath selector to find all a (anchor) elements with an attribute title that has the value "log out":

  find("xpath", "//a[@title='log out']")

• find_in_element(stype: SelectorType, sval: str, element: ElementList, index: int = 0) -> ElementList
  • stype: The selector type.
  • sval: The value of the selector type.
  • element: Find the web element inside one of these elements.
  • index: Choose from the list of matching elements (default 0, the first element).
  • return: A list of the web elements matching the selector stype with the value sval.

  This action finds web elements based on a selector type and the value of such a selector inside the given web element. This example returns a list of web elements with the name "filter" from inside a web element with the id "form":

  find_in_element("name", "filter", find("id", "form"), 0)

• get(stype: SelectorType, sval: str, index: int = 0) -> str
element_get(element: ElementList, index: int = 0) -> str
  • stype: The selector type.
  • sval: The value of the selector type.
  • element: A list of the web elements (typically from a find action).
  • index: Choose from the list of matching elements (default 0, the first element).
  • return: The value or text of a web elements matching the selector stype with the value sval.

  Get the value or text of a web element matching the selector stype with the value sval. An example where we get the value or text of an element with the id "about":

  get("id", "about")

• get_value(stype: SelectorType, sval: str, index: int = 0) -> str
element_get_value(element: ElementList, index: int = 0) -> str
  • stype: The selector type.
  • sval: The value of the selector type.
  • element: A list of the web elements (typically from a find action).
  • index: Choose from the list of matching elements (default 0, the first element).
  • return: The value of a web elements matching the selector stype with the value sval.

  Get the value of a web element matching the selector stype with the value sval. An example where we get the value of an element with the name "aggregate":

  get_value("name", "aggregate")

• get_text(stype: SelectorType, sval: str, index: int = 0) -> str
element_get_text(element: ElementList, index: int = 0) -> str
  • stype: The selector type.
  • sval: The value of the selector type.
  • element: A list of the web elements (typically from a find action).
  • index: Choose from the list of matching elements (default 0, the first element).
  • return: The text of a web elements matching the selector stype with the value sval.

  Get the text of a web element matching the selector stype with the value sval. An example where we get the text of the third element (at index 2) with the tag "h2":

  get_text("tag", "h2", 2)

• is_checked(stype: SelectorType, sval: str, index: int = 0) -> str
element_is_checked(element: ElementList, index: int = 0) -> str
  • stype: The selector type.
  • sval: The value of the selector type.
  • element: A list of the web elements (typically from a find action).
  • index: Choose from the list of matching elements (default 0, the first element).
  • return: The true if a web elements matching the selector stype with the value sval is checked.

  Check if a web element matching the selector stype with the value sval is checked. An example where we check if the checkbox with the name "checkbox1" is checked:

  is_checked("name", "checkbox1")

• check(stype: SelectorType, sval: str, index: int = 0)
element_check(element: ElementList, index: int = 0)
  • stype: The selector type.
  • sval: The value of the selector type.
  • element: A list of the web elements (typically from a find action).
  • index: Choose from the list of matching elements (default 0, the first element).

  Check a web element matching the selector stype with the value sval. This example checks the fourth checkbox on the web page (with index 3):

  check("xpath", "//input[@type='checkbox']", 3)

• uncheck(stype: SelectorType, sval: str, index: int = 0)
element_uncheck(element: ElementList, index: int = 0)
  • stype: The selector type.
  • sval: The value of the selector type.
  • element: A list of the web elements (typically from a find action).
  • index: Choose from the list of matching elements (default 0, the first element).

  Uncheck a web element matching the selector stype with the value sval. This example unchecks a web element with id "include-comments":

  uncheck("id", "include-comments")

• clear(stype: SelectorType, sval: str, index: int = 0)
element_clear(element: ElementList, index: int = 0)
  • stype: The selector type.
  • sval: The value of the selector type.
  • element: A list of the web elements (typically from a find action).
  • index: Choose from the list of matching elements (default 0, the first element)

  Reset the field value of a web element matching the selector stype with the value sval. This example clears a field with the name "search":

  clear("name", "search")

• select(val: str, stype: SelectorType, sval: str, index: int = 0)
element_select(val: str, element: ElementList, index: int = 0)
  • val: The value to select.
  • stype: The selector type.
  • sval: The value of the selector type.
  • element: A list of the web elements (typically from a find action).
  • index: Choose from the list of matching elements (default 0, the first element)

  Select the given value val in a web element matching the selector stype with the value sval. In this example, "year" is selected in the web element with the name "type":

  select("year", "name", "type")

• fill(val: str, stype: SelectorType, sval: str, index: int = 0)
element_fill(val: str, element: ElementList, index: int = 0)
  • val: The value to fill-in.
  • stype: The selector type.
  • sval: The value of the selector type.
  • element: A list of the web elements (typically from a find action).
  • index: Choose from the list of matching elements (default 0, the first element)

  Fill in the value val in a web element matching the selector stype with the value sval. An example where a web element with the name "search" is filled with the text "Python":

  fill("Python", "name", "search")

• scroll_to(stype: SelectorType, sval: str, index: int = 0)
element_scroll_to(element: ElementList, index: int = 0)
  • stype: The selector type.
  • sval: The value of the selector type.
  • element: A list of the web elements (typically from a find action).
  • index: Choose from the list of matching elements (default 0, the first element)

  Scroll to a web element matching the selector stype with the value sval. In this example, the view is scrolled to a div element with a class attribute having the value "signature":

  scroll_to("xpath", "//div[@class='signature']")

• click(stype: SelectorType, sval: str, index: int = 0)
element_click(element: ElementList, index: int = 0)
  • stype: The selector type.
  • sval: The value of the selector type.
  • element: A list of the web elements (typically from a find action).
  • index: Choose from the list of matching elements (default 0, the first element)

  Click on a web element matching the selector stype with the value sval. This example clicks on a web element with the text "OK" (typically a button):

  click("text", "OK")

• find_link(ltype: LinkType, lval: str) -> ElementList
  • ltype: The link selector type.
  • lval: The value of the link selector type.
  • return: A list of matching link elements.

  The action find_link returns web elements based on a link selector type and the value of such a link selector. This example returns a list of link elements with href attribute values containing "filter":

  find_link("partial_href", "filter")

• click_link(ltype: LinkType, lval: str, index: int = 0)
element_click_link(element: ElementList, index: int = 0)
  • ltype: The link selector type.
  • lval: The value of the link selector type.
  • element: A list of the web elements (typically from a find action).

  Click on a link element matching the selector ltype with the value lval. This example clicks on a link element with the partial text "news"

  click_link("partial_text", "news")

• is_present(stype: SelectorType, sval: str, wait_time: int | None = None) -> bool
  • stype: The selector type.
  • sval: The value of the selector type.
  • wait_time: How long to wait for the web element to be present (default None).
  • return: Returns True if the web element is present.

  The action is_present checks if a web element based on the selector type stype with the value sval is present. This example returns True if a web element with id "news" is present:

  is_present("id", "news")

• is_not_present(stype: SelectorType, sval: str, wait_time: int | None = None) -> bool
  • stype: The selector type.
  • sval: The value of the selector type.
  • wait_time: How long to wait for the web element to be present (default None).
  • return: Returns True if the web element is not present.

  The action is_not_present checks if a web element based on the selector type stype with the value sval is not present. This example returns True if a web element with name "loginform" is not present:

  is_not_present("name", "loginform")

• is_text_present(text: str, wait_time: int | None = None) -> bool
  • text: The text to find.
  • wait_time: How long to wait for the text to be present (default None).
  • return: Returns True if the text is present.

  The action is_text_present checks if the text is present. This example returns True if the text "Login succeeded" is present within 3 seconds:

  is_text_present("Login succeeded", 3)

• is_text_not_present(text: str, wait_time: int | None = None) -> bool
  • text: The text that should't be present.
  • wait_time: How long to wait for the text to be present (default None).
  • return: Returns True if the text is not present.

  The action is_text_not_present checks if the text is not present. This example returns True if the text "Login failed" is not present:

  is_text_not_present("Login failed")

• doall(stype: SelectorType, sval: str, action: ActionFunc, *args: list, sep: str = "\n", **kw: dict)
element_doall(elements: ElementList, action: ActionFunc, *args: list, sep: str = "\n", **kw: dict)
  • stype: The selector type.
  • sval: The value of the selector type.
  • elements: A list of the web elements (typically from a find action).
  • action: .
  • args: .
  • sep: (default "\n").
  • kw: .

  Do the action on all web elements. In this example, check all the checkbox input elements:

  doall("xpath", "//input[@type='checkbox']", element_check)

• cond(condition: bool, stype: SelectorType, sval: str, ifaction: ActionFunc, *args: list, index: int = 0, elseaction: ActionFunc = None, **kw: dict)
element_cond(condition: bool, elements: ElementList, ifaction: ActionFunc, *args: list, index: int = 0, elseaction: ActionFunc = None, **kw: dict)
  • condition: The condition.
  • stype: The selector type.
  • sval: The value of the selector type.
  • elements: A list of the web elements (typically from a find action).
  • ifaction: The action performed if condition is true.
  • args: Arguments to the action (both ifaction and elseaction).
  • elseaction: The action performed if condition is false (default None).
  • index: Choose from the list of matching elements (default 0, the first element)
  • kw: Keyword arguments to the action (both ifaction and elseaction).

  Do the ifaction if condition is True. Do the elseaction if it is provided and condition is False. In this example, the checkbox named "checkbox1" is toggled:

  cond(verify(is_checked("name", "checkbox1"), True, assert_it = False), \
  "name", "checkbox1", ifaction = element_uncheck, elseaction = element_check)

• attach_file(file_path: str, stype: SelectorType, sval: str, index: int = 0)
element_attach_file(file_path: str, element: ElementList, index: int = 0)
  • file_path: The absoulute path to the file.
  • stype: The selector type.
  • sval: The value of the selector type.
  • element: A list of the web elements (typically from a find action).
  • index: Choose from the list of matching elements (default 0, the first element)

  Attach a file to a web element (a file input element). In this example, the file "/path/to/file" is attached to a web element with the name "thefile":

  attach_file("/path/to/file", "name", "thefile")

Above, you can see that almost all actions with an argument pair SelectorType stype and its value sval have a version where an ElementList is the argument instead of these two arguments (this is also true for actions with a LinkType/link-value argument pair). The exceptions are actions that only search for elements (e.g., find and is present actions). An example of an action with an ElementList version is the fill action. It has the companion element_fill. The element_ versions of these actions can be combined with variables set to web elements, or actions returning web elements. The two lines below perform the same interaction with a web page (fill in the text "LaTeX" in the web element with the id "searchtext"):

fill("LaTeX", "id", "searchtext")
element_fill("LaTeX", find("id", "searchtext"))

In the listing of the web interaction actions from the webinteract module, the following non-standard Python types were used in the type hints:

• SelectorType One of the following text strings: "css", "id", "name", "tag", "text", "value", or "xpath".
• LinkType One of the following text strings: "text", "partial_text", "href", or "partial_href".
• ElementList A list of web elements. In many cases, this is a list with a single web element.
• ActionFunc A web interaction action (e.g., element_check)

Comments and troubleshooting

When executing wia scripts, a lot of different software and systems are involved. This includes a web browser and a matching WebDriver (a remote control interface that enables introspection and control of user agents), Python 3, a keychain, and so on. This section includes comments and hints for maintaining and troubleshooting this software complexity.

ChromeDriver

If the webinteract module is used with the chrome driver, the Chrome browser and a matching version of ChromeDriver have to be installed. To ensure that the Chrome browser is up-to-date, go to the About page of the browser. To ensure that the ChromeDriver is up-to-date on my Mac, I install it and keep it updated using Homebrew:

brew install chromedriver
pip3 install selenium
chromdriver		# run "chromedriver" once to get system approval

On a Mac, you might run into a problem since the ChromeDriver is not properly signed:

To solve this, we have to make an exception for chromedriver (if we trust the installed chromedriver program). In macOS 14 or older: locate the chromedriver program file in Finder, control-click on the file, choose Open from the shortcut menu, and click the Open button on the pop-up. In macOS 15 or newer: Open System Settings, choose Privacy & Security in the left panel, scroll down to the Security section, and press the Open Anyway button on the right side of the text about blocking ChromeDriver (for this to appear, you have to run chromedriver at least once):

You might have to do this every time the Chrome Browser and ChromeDriver is updated.

In the steps above installing ChromeDriver, I also installed the Python module selenium. This is a Python binding for WebDrivers, and it supports Chrome and ChromeDriver. The splinter framework uses the selenium module.

Keychains and the keyring module

The keyring module supports macOS Keychain, Freedesktop Secret Service (requires SecretStorage), KDE4/KDE5 KWallet (requires dbus), and Windows Credential Locker. Other keyring services are available through third-party backends. I have only tested the webinteract module with the macOS Keychain.

When using macOS, you can use the built-in command line tool security to add items to the macOS Keychain. If you want to add a password for the examples above to the macOS keychain, you can use this command:

security add-generic-password -a my@email.address -s email.provider -w

The -w flag at the end makes the security tool prompt you for the password.