What about voice commands - Page 2

What about voice commands

Various things about Spring that do not fit in any of the other forums listed below, including forum rules.

Moderator: Moderators

User avatar
momfreeek
Posts: 625
Joined: 29 Apr 2008, 16:50

Re: What about voice commands

Post by momfreeek »

Jools wrote:I was thinking of leaving that open to the user. Just intergrate SR with lua and creativity will spring up with something. How about
"nap commander" instead of
ÔÇö Find and select nearest transport
ÔÇö Find enemy commander
ÔÇö to to load enemy commander

(all of the above requires user interaction, whereas the nap command could be run independently)
This could be created already and bound to a key/on-screen button.

If its not been done then its probably very difficult or not very useful (timing and micro is everything). If it has been done then you are just linking existing commands to a voice command much like binding a key.
User avatar
AF
AI Developer
Posts: 20687
Joined: 14 Sep 2004, 11:32

Re: What about voice commands

Post by AF »

"select 5 stumpy"
There are 120 stumpies on the battlefield, do you mean the 5 nearest to the cursor? To the focal point of the screen? 5 idle stumpies? All 120 are in a giant blob, shall I select the ones right in the middle and have them spend 10 minutes pushing their way out? Wait they're now in groups of 5 but there are 40 such groups onscreen, which one do you want to select?
"select vehicle lab"
The nearest idle lab? Nearest to the selected unit? Mouse position? Or do you want the nearest lab whether its building or not? This lab here has only one thing left in its queue, do you mean the advanced vehicle lab or the amphibious vehicle lab? You have captured the other factions tech, did you mean the arm or the core vehicle lab you built?
"build 10 flash"
Where? Using a factory? Using an engineer? Which factory, which engineer?
momfreeek wrote:
Jools wrote:I was thinking of leaving that open to the user. Just intergrate SR with lua and creativity will spring up with something. How about
"nap commander" instead of
ÔÇö Find and select nearest transport
The weak Atlas 2 ticks away? Or the super armoured super fast transport 3 clicks away? Nearest transport to the focus of the camera? Nearest transport to the mouse cursor? Nearest transport to the selected unit? You have 3 selected, which one should it check for the closest to?
ÔÇö Find enemy commander
Its a 3v3, should I load commander 1, 2, or 3?
ÔÇö to to load enemy commander
Which commander? The exposed com owned by the most powerful player 2 ticks away? Or the nearest com 1 tick away surrounded by nothing but flak cannons with no factories or other units?
(all of the above requires user interaction, whereas the nap command could be run independently)
This could be created already and bound to a key/on-screen button.

If its not been done then its probably very difficult or not very useful (timing and micro is everything). If it has been done then you are just linking existing commands to a voice command much like binding a key.
User avatar
Anarchid
Posts: 1384
Joined: 30 Nov 2008, 04:31

Re: What about voice commands

Post by Anarchid »

The way i'd go on this:
There are 120 stumpies on the battlefield, do you mean the 5 nearest to the cursor? To the focal point of the screen? 5 idle stumpies? All 120 are in a giant blob, shall I select the ones right in the middle and have them spend 10 minutes pushing their way out? Wait they're now in groups of 5 but there are 40 such groups onscreen, which one do you want to select?
syntax: select [integer_word|all|half|third|other_quantity_word] [location_word] [status_word] [...other filter word] unittype

example: 'select stumpy'
result: select all stumpies (quantifier defaults to 'all' if integer or some other quantifier is not provided)

example: 'select five stumpy'
result: select any five stumpies; which ones is undefined, user didnt specify = user doesnt care

example: 'select vehicle-lab'
result: select all vehicle labs

example: 'select idle factory'
result: select all idle factories of all kinds (basically factory here is a unittype tag that matches any factory regardless of what it makes, what faction, etc)

example: 'select five nearest contagious idle stumpy'
result: select five stumpies that are in a 'single block' (no other stumpies between them) that are closest to the mouse cursor and are idle.

... etc.

Come on, if EndWar can have this (maybe somehow different though), so can Spring. Just a case of writing up an appropriate command language (probably best used mod-specific, so i'd say leave this to Lua)
User avatar
momfreeek
Posts: 625
Joined: 29 Apr 2008, 16:50

Re: What about voice commands

Post by momfreeek »

AF wrote:
"select 5 stumpy"
There are 120 stumpies on the battlefield, do you mean the 5 nearest to the cursor?
yes.
AF wrote:Shall I select the ones right in the middle and have them spend 10 minutes pushing their way out?
No, select the 5 nearest to the cursor.
Wait they're now in groups of 5 but there are 40 such groups onscreen, which one do you want to select?[/color]
the 5 nearest to the cursor!

AF wrote:"select vehicle lab"
The nearest idle lab? Nearest to the selected unit? Mouse position? Or do you want the nearest lab whether its building or not? This lab here has only one thing left in its queue, do you mean the advanced vehicle lab or the amphibious vehicle lab? You have captured the other factions tech, did you mean the arm or the core vehicle lab you built?
There's probably only 1. select the closest. If I meant amphibious I'd say amphibious. If I wanted all I'd say "all".

I mean.. there's some ambiguity but its not difficult to work out precedence that the player will quickly understand. Selecting between many could perhaps be achieved with further selectors (but hey, why not start simple).
AF wrote:"build 10 flash"
Where? Using a factory? Using an engineer? Which factory, which engineer?
using the factory i just selected when I said "select vehicle factory".

Building from unselected factories could probably be achieved too (but again, why not start simple).

I figured the obvious solutions were implicit in the suggestion.
Last edited by momfreeek on 27 Oct 2011, 15:53, edited 2 times in total.
User avatar
SpliFF
Posts: 1224
Joined: 28 Jul 2008, 06:51

Re: What about voice commands

Post by SpliFF »

The next person to use the words 'easy' or 'simple' in conjunction with any aspect of voice control is getting added to my ignore list. :roll:

It's only simple until you actually think about it.
User avatar
momfreeek
Posts: 625
Joined: 29 Apr 2008, 16:50

Re: What about voice commands

Post by momfreeek »

well.. choosing between a number of labs/units is simple if you follow simple rules that the player will easily understand (rather than building an ai for it).

Obviously I am assuming the voice recognition part is implemented (I'm not saying that part is simple). I guess the trouble with sentences is determining when its finished. "select 5 stumpies.. now" ?
User avatar
Jools
XTA Developer
Posts: 2816
Joined: 23 Feb 2009, 16:29

Re: What about voice commands

Post by Jools »

AF wrote:ÔÇö Find and select nearest transport
The weak Atlas 2 ticks away? Or the super armoured super fast transport 3 clicks away? Nearest transport to the focus of the camera? Nearest transport to the mouse cursor? Nearest transport to the selected unit? You have 3 selected, which one should it check for the closest to?
I was using that example to illustrate the way you need to think with vocie UI, you need to think on what you want to accomplish, not how it's done using conventional UI:s. There would be little point of using voice as a means to provide a shortcut to something that's done with the mouse or the keyboard, since those things are then done best using those means. So using a voice command such as "press enter key" is stupid, in that case, just press enter key instead.

But since you asked, I will explain the rationale in the widget I referred to (I just used it as an example, didn't mean to imply that it's useful).
ÔÇö Find and select nearest transport
- select all transports. Give bonus score for airborne ones, multiply with hp/maxhp. Add points if user has put the unit in a group. Divide by distance to selected unit. Sort by total score. Select highest scorer.

That widget loads the selected unit into best transport. It only works with 1 unit selected currently. It could be modified to order a nap of enemy commander, if someone sees the point in that (I don't). I was making an example.
User avatar
AF
AI Developer
Posts: 20687
Joined: 14 Sep 2004, 11:32

Re: What about voice commands

Post by AF »

you need an AI for even the most primitive of commands.

And sure you can say that you would have said amphibious if you meant it but not everyone is so precise and in the time you took to say it out loud twice and cancel the command and redo it'd have been easier to do it via mouse and keyboard anyway.

Also what are you to do if you say select the closest fido? What if the cursor is on top of a transport containing 5 fidos?

There's the command you gave, then there's what you expect to happen, and that won't fit everyones expectations.

Even when you have a clearcut unambiguous command, there're always additional things to do that you expect but never asked for because its obvious,

e.g. in Shard, it attacks the weakest part of the enemies base, seems obvious, it works, Shard does as it's told, and in doing so it walks through the most heavily defended part to get there, defeating the point of it all, precisely because it was given a blind command to move to the target and attack anything there.
User avatar
momfreeek
Posts: 625
Joined: 29 Apr 2008, 16:50

Re: What about voice commands

Post by momfreeek »

AF wrote:And sure you can say that you would have said amphibious if you meant it but not everyone is so precise and in the time you took to say it out loud twice and cancel the command and redo it'd have been easier to do it via mouse and keyboard anyway.
Some learning the interface woudl clearly be involved. This seems like a general problem with voice commands and interfaces in general.
Also what are you to do if you say select the closest fido? What if the cursor is on top of a transport containing 5 fidos?
ignore units inside transport (simple rule will do)
There's the command you gave, then there's what you expect to happen, and that won't fit everyones expectations.
Again, learning the interface and how it implements your commands. This is just as true for the existing interface: pathfinding, "hold ground" vs "roam" etc.

Simple rules are easier for a human to learn, understand and use even if they do not initially do what you might hope.
Last edited by momfreeek on 27 Oct 2011, 18:01, edited 1 time in total.
User avatar
knorke
Posts: 7971
Joined: 22 Feb 2006, 01:02

Re: What about voice commands

Post by knorke »

one could "just" link all the hotkeys to voice commands.
So instead of pressing ctrl+c you say "select commander", screaming "EXPODE!!!" is the same as pressing ctrl+d, "turn off" is pressing x and so on.
The UT video is not much more than that, instead of pressing "r" he says "rocket launcher" to select that weapon. He does not say "shot rocket launcher through left window."
User avatar
Jools
XTA Developer
Posts: 2816
Joined: 23 Feb 2009, 16:29

Re: What about voice commands

Post by Jools »

In that case, it's already possible with WSR: there's a program called WSRMacros which lets you send a keystroke on a voice recognition. Keystroke can be such as Ctrl+Alt+Insert+R, and you can filter it to only work when spring.exe is in focus.

Incidentally, Siri uses Nuance Dictation, which is the company behind Dragon Naturally Speeching. DNS is better than WSR on dictation and recognises more common names, but WSR integrates better with Windows (surprise!). I believe there is an API for DNS too which can be used to code macros.
luckywaldo7
Posts: 1398
Joined: 17 Sep 2008, 04:36

Re: What about voice commands

Post by luckywaldo7 »

AF wrote:The issue is not voice recognition, Google have APIs for that.

The issue is now that we have a plaintext natural language input what are we to do? AIs already have a lot of these issues and still haven't satisfied.

For example:

"Attack the enemy"
  • With what?
  • Where shall I attack the enemy
  • From which direction
  • Point blank or at a distance
  • Which enemy should be hit first
  • How do you get there? ( the direct route isn't always the optimum
  • In a line? In a surrounding circle? Pincer movement?
"Withdraw units"
  • Which units?
  • How far should we withdraw?
  • Where should they go to?
  • Are they withdrawing from their current position or from a targetted position?
etc etc etc, and this is all after you've figured out the meaning of what is being said.


Any voice recognition is going to be limited in what it can do, most likely single or two word commands issued to what is currently selected such as "cloak" or "self destruct". Commands such as attack wouldn't issue the order, they would be the same as pressing the attack button or A, you'd still need to click somewhere.
^ This is what I meant when I said it would be hard to match the precision of keyboard and mouse. It will be hard to make the connection between inputting spoken commands and translating into units doing what you want them to do.
knorke wrote:one could "just" link all the hotkeys to voice commands.
So instead of pressing ctrl+c you say "select commander", screaming "EXPODE!!!" is the same as pressing ctrl+d, "turn off" is pressing x and so on.
The UT video is not much more than that, instead of pressing "r" he says "rocket launcher" to select that weapon. He does not say "shot rocket launcher through left window."
Well, he was talking about Siri, so I assume he meant something more advanced than that.
User avatar
momfreeek
Posts: 625
Joined: 29 Apr 2008, 16:50

Re: What about voice commands

Post by momfreeek »

luckywaldo7 wrote:Well, he was talking about Siri, so I assume he meant something more advanced than that.
to be fair the OP mentioned siri once (as what lead him to the idea of voice command) then he suggested exactly what knorke suggested.

No-one suggested replacing the mouse before the arguments against it. :?
User avatar
maackey
Posts: 490
Joined: 02 Jul 2008, 07:11

Re: What about voice commands

Post by maackey »

Jools: wow that really is nice. I was playing around just with unitgroups and selecting things, but I can imagine that voice activated shortcuts would be really nice for other things like jumping, switching units on/off, load/unload and other commands that would take your hand away from its "home" position.

Although... I never was able to get CTRL+c or other mixed keys to work :s What is the XML file supposed to look like? I've tried
<sendKeys>{{CTRL}}c</sendKeys>
<sendKeys>{CTRL}c</sendKeys>
<sendKeys>CTRL+c</sendKeys>
<sendKeys>CTRLc</sendKeys>
<sendKeys>^c</sendKeys>
and none of them seem to work... am I missing something else?

I was also trying to get "group [number]" to work but its much quicker to just make 10 separate macro files :/
User avatar
Jools
XTA Developer
Posts: 2816
Joined: 23 Feb 2009, 16:29

Re: What about voice commands

Post by Jools »

The wsr macro file has to be signed by windows in order for it to work. Sometimes you must re-sign them periodically. It's a security thing, and probably the most likely reason it didnt work for you.

There's a specification for the xml format, I can't find it now, but I'll post some examples:

1) Call spring user with skype:

Code: Select all

<?xml version="1.0" encoding="UTF-16"?>
<speechMacros>
  <command>
	<appIsInForeground processName="TASClient.exe"/>
	<appIsInForeground processName="spring.exe"/>
    <listenFor>Call ansik in spring</listenFor>
    <run command="C:\Program Files (x86)\Skype\Phone\Skype.exe" params="/callto:springansiktsburk"/>
  </command>

<Signature>
blablabla
</Signature>
</speechMacros>
2) What song is playing in winamp?

Code: Select all

<?xml version="1.0" encoding="UTF-16"?>
<speechMacros>

<!-- Read artist -->
<command>
	<listenFor>Song info</listenFor>
	<listenFor>Track info</listenFor>
	<listenFor>What's the name of this song?</listenFor>
	<listenFor>What music is playing?</listenFor>
	<listenFor>What song is this?</listenFor>
	<listenFor>Identify this music composition</listenFor>
	<waitFor seconds=".45"/> 
	<sendKeys>{{CTRL}}{{SHIFT}}{{ALT}}8</sendKeys>

	<script language="JScript">
	<![CDATA[
// Get the data out of the clipboard
	var text = Application.clipboardData.getData("text"); 
	Application.Speak(text);
	]]></script>
</command>
<Signature>
blablabla
</Signature>
</speechMacros>

I have mapped ctrl+shift+alt+8 in winamp as "copy song title to clipboard" or something like that.

To make a simple macro, just use the wizard:

http://archive.msdn.microsoft.com/wsrma ... s%20Wizard
User avatar
Jools
XTA Developer
Posts: 2816
Joined: 23 Feb 2009, 16:29

Re: What about voice commands

Post by Jools »

maackey wrote:
Although... I never was able to get CTRL+c or other mixed keys to work :s What is the XML file supposed to look like? I've tried
<sendKeys>{{CTRL}}c</sendKeys>
<sendKeys>{CTRL}c</sendKeys>
<sendKeys>CTRL+c</sendKeys>
<sendKeys>CTRLc</sendKeys>
<sendKeys>^c</sendKeys>
and none of them seem to work... am I missing something else?
It should be <sendKeys>{{CTRL}}c</sendKeys>

Modifier keys such as ctrl, alt use double brackets, some other category uses single brackers and simple keys uses none.
User avatar
Columbus
Posts: 158
Joined: 12 Jun 2006, 09:34

Re: What about voice commands

Post by Columbus »

Something as fancy as Siri would be cool, but at first, anything will do.

Basically, the command syntax should be quite fixed, with a specific order of parameters, since that is the most easiest to implement.
If an interface that can unfuddle these commands from the speech engine and relay them to a lua widget can be made, then everything will be ready to give developers free hands to create any features using this(the sky is the limit?).

The main point is, I guess, that you can do anything you want with it.
If this interface would be available, I'm sure some really awesome features would be made that use it(like the ones suggested in this thread), which makes it in general a good idea.

Didn't expect so much interest in this really :P
User avatar
bobthedinosaur
Blood & Steel Developer
Posts: 2702
Joined: 25 Aug 2004, 13:31

Re: What about voice commands

Post by bobthedinosaur »

the crappy End War rts had this years ago. http://www.youtube.com/watch?v=XQNCfSuQ ... re=related
User avatar
maackey
Posts: 490
Joined: 02 Jul 2008, 07:11

Re: What about voice commands

Post by maackey »

I've been playing around with voice commands and they are really neat. There is about 1 second delay between your command and the results, but its really cool to say, "commander, build shield bot" and the outline of the factory appears at your cursor tip.

Unfortunately I can't get any held down combos to work. eg. "join group [number]" and "commander" don't work at all and I don't know how to fix it.

Here is my springmacros.WSRMac file atm, its quite incomplete and still a work in progress.

Code: Select all

<?xml version="1.0" encoding="UTF-16"?>
<speechMacros>
	<appIsRunning processName = "spring.exe"/> 
	<command>
		<listenFor>group [number]</listenFor>
		<numbers name="number" start="1" stop="10" />
		<sendKeys>{[number]}</sendKeys>
	</command>
	
	<command>
		<listenFor>join group [number]</listenFor>
		<numbers name="number2" start="1" stop="10" />
		<sendKeys>{{CTRL}}{[number]}</sendKeys>
	</command>
	
	<command>
		<listenFor>build</listenFor>
		<sendKeys>b</sendKeys>
	</command>
	
	<command>
		<listenFor>build [factory]</listenFor>
		<rule name="factory">
		<list propname="shortcut">
			<p valstr="b{30 WAIT}a{30 WAIT}a">cloaky bot</p>
			<p valstr="b{30 WAIT}a{30 WAIT}w">shield bot</p>
			<p valstr="b{30 WAIT}a{30 WAIT}e">jumpy bot</p>
			<p valstr="b{30 WAIT}a{30 WAIT}x">vehicles</p>
			<p valstr="b{30 WAIT}a{30 WAIT}c">tanks</p>
			<p valstr="b{30 WAIT}a{30 WAIT}z">hovers</p>
			<p valstr="b{30 WAIT}a{30 WAIT}q">spiders</p>
			<p valstr="b{30 WAIT}q{30 WAIT}q">planes</p>
			<p valstr="b{30 WAIT}q{30 WAIT}d">gunships</p>
			<p valstr="b{30 WAIT}q{30 WAIT}x">navy</p>
			<p valstr="b{30 WAIT}q{30 WAIT}a">athena</p>
			<p valstr="b{30 WAIT}q{30 WAIT}w">pad</p>
		</list>
		</rule>
		<sendKeys>{[factory.shortcut]}</sendKeys>
	</command>
	
	<command>
		<listenFor>cloak</listenFor>
		<sendKeys>k</sendKeys>
	</command>
			
	<command>
		<listenFor>jump</listenFor>
		<sendKeys>j</sendKeys>
	</command>
	
	<command>
		<listenFor>load</listenFor>
		<sendKeys>l</sendKeys>
	</command>
	
	<command>
		<listenFor>unload</listenFor>
		<sendKeys>u</sendKeys>
	</command>
	
	<command>
		<listenFor>screenshot</listenFor>
		<sendKeys>{F12}</sendKeys>
	</command>
	
	<command>
		<listenFor>commander</listenFor>
		<sendKeys>{{CTRL}}c</sendKeys>
	</command>

<Signature>
MIIDnQYJKoZIhvcNAQcCoIIDjjCCA4oCAQExCzAJBgUrDgMCGgUAMGcGCisGAQQBgjcCAQSgWTBX
MDIGCisGAQQBgjcCAR4wJAIBAQQQQf9itGlsy0O8AOWC49jANAIBAAIBAAIBAAIBAAIBADAhMAkG
BSsOAwIaBQAEFELtqi3eAQyXihlCgaD++IhnmRcMoIIBzTCCAckwggEyoAMCAQICEDZ96ySQsde1
SO5jIfFXGJswDQYJKoZIhvcNAQEFBQAwEDEOMAwGA1UEAxMFQ2hyaXMwHhcNMTExMDI3MjA0NzMz
WhcNMTIxMDI3MDI0NzMzWjAQMQ4wDAYDVQQDEwVDaHJpczCBnzANBgkqhkiG9w0BAQEFAAOBjQAw
gYkCgYEAp4WzNEG7Moa8dXjP1V1VLoCEvrstOfLZnazNBXMLP4/7fXUNpgGxDo2GgRijTgbsLxBF
jOP/rtaB1fb7kPy3avIt8p+Iv8M8rsNKxquJaGIqQuVXP57J745d2RQ3GJdzeFw3pMrBD6tUkCsT
CsxAIM6oTQ2MQVaW6EiIJ+4KwQcCAwEAAaMkMCIwCwYDVR0PBAQDAgeAMBMGA1UdJQQMMAoGCCsG
AQUFBwMDMA0GCSqGSIb3DQEBBQUAA4GBAHa73udHcW/dpa+jguzJQfwb7jmJxccaPteaPlBBAlFs
nwfCHPbH8YZOwC1UoVjvrc5MV4vE2u6UGxlH+290DakFfIS7rQHXoLsrNKRoVNe2m27JdC81Q0rl
vburvIaM8uoUjCAmTJ4eXH52rnry0iztWZNs1XO4ubW+0IxikiNYMYIBPDCCATgCAQEwJDAQMQ4w
DAYDVQQDEwVDaHJpcwIQNn3rJJCx17VI7mMh8VcYmzAJBgUrDgMCGgUAoHAwEAYKKwYBBAGCNwIB
DDECMAAwGQYJKoZIhvcNAQkDMQwGCisGAQQBgjcCAQQwHAYKKwYBBAGCNwIBCzEOMAwGCisGAQQB
gjcCARUwIwYJKoZIhvcNAQkEMRYEFDEDnOWLCJVdIJNAo1dnPph/1mJDMA0GCSqGSIb3DQEBAQUA
BIGAXwBOQ175xvOZcuIuke8BFws77/3QYjrkOpBu6ztrzFmO0fDDWxanHKVDZzBGW/2GQHouIdxm
1pRDkRBm8mf8xZUK5rwuJIC87SGqT+yAbmPuolywFr+DwMwojJkvOmnqVdxxAuiotZH+PA8Nl47n
jKhaq2rrm5HJNqXXlIPjpXQ=
</Signature>
</speechMacros>
User avatar
Jools
XTA Developer
Posts: 2816
Joined: 23 Feb 2009, 16:29

Re: What about voice commands

Post by Jools »

Cool, doesn't the speech recognition use plenty of system resources? It does for me, and especially if you run Mozilla products at the same time (I mean, Opera runs normally but Firefox and Thunderbird slow down a lot when SR is listening).

Why do you have it waiting there? I just put a wait so that winamp would have time to copy the song info, and I experimented so that 0.45 secs would be best. Also, I thought it sounded more natural if the answer wasnt immediate.

I could think of a macro where you define some screen coordinates, and then say "move all hawks to south-east corner". Something like that, it would save a lot of multitasking time, which is better used microing your commander.
Post Reply

Return to “General Discussion”