UI scripting avec python

[hugome]

On m'a demandé comment je faisais pour scripter une appli en utilisant du python open source

Je donne ici l'exemple de Spotify qui n'a pas créé de dictionnaire AppleScript et n'est pas scriptable en direct
J'ai un raccourci géré par Spark (programme qui permet d'attribuer une touche de raccourci à un AppleScript)
Voici l'AppleScript :
À noter, ça marche tant que Spotify ne change pas son UI

use AppleScript version "2.4" -- Yosemite (10.10) or later
use scripting additions

# purpose:
-- script for liking the current played spotify song
-- if the songs is already liked, it used to be toggled. (Not working anymore)

# configuration:
-- pyautogui should be installed
-- to install, type in terminal the following line:
-- python3 -m pip install pyautogui 
-- to check if installed
-- pip3 show pyautogui

# activation:
-- Spark shortcut
-- Spark needs to be updated if script modified

# program body

-- check your python path and version
set pythonPath to "/usr/local/bin/python3"

-- get the position of shuffle icon in spotify UI with cmd shift 4
-- window needs to be full screen or nearly so
-- move 1 cm to the left
-- report number here
set titleX to 1146
set titleY to 1381

activate application "Spotify"

-- script to select (& click) next to the shuffle icon
set theScript to "
" & pythonPath & "<< END
import pyautogui
x, y = pyautogui.position()                     # Save current mouse position
pyautogui.click(" & titleX & "," & titleY & ")    # Click on title
pyautogui.moveTo(x, y)                     # Restore mouse position
END"

do shell script theScript # select next to shuffle icon

tell application "System Events" to tell process "Spotify"
    
    keystroke tab using {shift down} # select heart icon
    keystroke return # performs the like
    delay 1 # for visual confirmation
    keystroke tab using {command down} # go back to former application
end tell