Two Panes under Window both without Name or AutomationID

Jan 1, 2013 at 5:26 PM

I was just looking through the Cmdlets, and I don't see an "Index" Parameter.  I believe it would be helpful if I could state the following:

Get-Process whatever | Get-UIAWindow | Get-UIAPane -Index 2 | ...

This way, I could specify exactly which pane I want to grab instead of having to iterate through all of the panes looking for the correct one.  This is of course, just a suggestion.  I have only been using these cmdlets for a couple of days, and already I am extremely impressed with the functionality and breadth of this Extension for powershell.  I had been toying around with making powershell functions that utilized the UIAutomation class, and this just completely thwarted any idea of "progress" that I had. 

To be clear, I know I can get to the target control with:

Get-Process whatever | Get-UIAWindow | Get-UIAPane | ...

This just appeared to take a few seconds, and I was thinking with the specification of the index there may be no need to iterate through all of the objects.  Thanks for all of your hard work on this extension; it is simple to use, and has an extreme amount of functionality.

Jan 1, 2013 at 9:39 PM

Hello jbartels,

with one your post you have raised several questions at a time. I'll try to separate them from each other and this might help you, I hope.

1) first of all, indexing of the result is what you can easily do by PowerShell itself:

(Start-Process calc -PassThru | Get-UIAWindow | Get-UIAButton -n [1-3])[1] | Read-UIAControlName

This should return '2' as buttons on the main page of Calculator never change their order (testers who tested Calculator are really happy people :)).

2) now about iterating the UIAutomation tree and, therefore, the performance.

How MS UI Automation works (I don't know exactly how, of course, this is just results of my observation) we can imagine as walk through the graph. An application with GUI is a tree with branches of various length (I'd recommend to try Microsoft's UIA Verify from codeplex to look at this). If we say "go and return to us all the panes the app has", MS UI Automation walks through every branch in the UIAutomation tree.

If your application has tens or hundreds of controls, it is a problem, sometimes a big problem. Hundreds of controls are not so rare situation as it seems: a grid, listview or a web page are not the complete list of examples.

We can't (now) do anything to this algorithm, however, sometimes, we can be smarter than this.

2.1) If you have something, just anything what is between window and the pane of your interest, you can direct the tree walker this way (Get-UIAWindow | Get-.... | Get-.... | Get-UIAPane).

2.2) A 'shourt cut' way can be made via the -Win32 parameter (some controls have so-called handles, and, wow, old technologies rules!, search among cotrols with handle is much faster than by MS UI Automation calculations.

Get-UIAWindow | Get-UIAButton -n Next -Win32

These cmdlets have been used for end-to-end automation of a real application with 3rdparty grid, and the default state of grid on a certain page took up to fifteen minutes to access a grid cell (a control that has no handle). On the opposite, access to links and buttons works instantly, as they have handles.

2.3) We have another way to regulate the speed: user settings for three ways to search

[UIAutomation.Preferences]::DisableWildCardSearch # default: $false
[UIAutomation.Preferences]::DisableWin32Search # default: $false
[UIAutomation.Preferences]::DisableExactSearch # default: $true

The first way is a search via collecting all controls of the type(s) selected by user and searching by control's properties in the collection - this is the default search. Not surprisingly, it can be slow when we have a number of controls.

The second way is a search via handles (it's useless in situations when we exactly know that we need handless contol(s)).

The third way is the original way to search - just how MS UI Automation searches (no wildcards). You may switch off two first way and turn on the third to check what is better.

2.4) The first way of search has an additional capability for complicated cases: it consumes hashtables.

Shortly, hashtables of properties look like:

Start-Process calc -PassThru | Get-UIAWindow | Get-UIAButton -n [1-3] | ConvertTo-UIASearchCriteria
# @{Name="1";AutomationId="131";ControlType="Button";}
# @{Name="2";AutomationId="132";ControlType="Button";}
# @{Name="3";AutomationId="133";ControlType="Button";}

To get the full set of properties (including pid and handles that are never useful on the second run of the app):

Start-Process calc -PassThru | Get-UIAWindow | Get-UIAButton -n [1-3] | ConvertTo-UIASearchCriteria
You are able to combine these properties in any order, not forgetting to separate them with semicolon and to include the value (of any type) into quotes.

 Start-Process calc -PassThru | Get-UIAWindow | Get-UIAControl -SearchCriteria @{accesskey="Alt+V"} | Read-UIAControlType

This code returns a menu item (Calculator lacks controls with access keys and accelerator keys).

3) Sometimes it's unavoidable to click to a control by coordinates. For example, if a control of your interest never changes the position, you could try something like:

Get-UIAWindow | Move-UIACursor -X 100 -Y 200 | Get-UIAControlFromPoint


Get-UIAWindow | Invoke-UIAControlClick -X 50 -Y 60 | Get-UIAControlFromPoint

The Invoke-UIAControlClick cmdlet accepts a control and clicks if the control has handle. If not, it get its parent, even the parent of parent and so on, unless a control with handle is gotten. After that it calculates the position relatively to a control with handle and clicks pretty well to what we need to.

4) If you are using some tool that returns handles of controls, you can use the cmdlet Get-UIAControlFromHandle

Jan 21, 2013 at 10:51 PM

I forgot to mention one more option: you may use 'siblings' cmdlets: Get-UIAControlNextSibling and Get-UIAControlPreviousSibling.