31 replies to this topic

[akingcoder]

Since I really do want to get nub support working for non-jailbroken iPhones (and hence, App Store approval), I spent some time and extended InfiniteSands iControlPad-Tester code to add support for nubs. I've forked his github repo and committed all my changes https://github.com/a...ntrolPad-Tester - if you have a Mac, it'll also work in the simulator if you want to test it out. Note: this works with the "special packet mode" when using HID Keyboard (A+X on startup)

I did make a quick 1 min video of it just to show it working and dumped it up on youtube:




It is actually working fairly well - I think with just a few more tweaks from MWeston, it could be a workable solution. I'd expect you could even get apps into the App store that could use this as an alternate control scheme (would have to support touch screen controls as the initial input controls).

The main issues I've encountered so far are:

• Sometimes, letting go of all buttons and nubs, the iControlPad will continuously send a stream of 'w' (or occasionally, other constant values). ie: here's an example when just trying the iCP out in textedit under OSX:
  

  wX8&w>>w7wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww\B@@

  Just touching the nub will stop this stream
• Occasionally, it appears that the iCP will drop a nub value from the 4 values passed
• Moving a nub fully to the left seems to stop the other nub from reporting any values until you pull back a bit on the other nub.

I really would like to add iControlPad support to my 3D iOS App - and it does appear so close! If there's anything else I can do to help get this fully working, please let me know. A couple of suggestions to reduce the number of characters sent:

• instead of a [-32, 32] X/Y range, a [-7, 8] range might be better - (even [-4,4] would work for what I need)
• instead of ('w' lx ly rx ry), why not just keep streaming the x/y coords without the 'w' until some other button event occurs or releasing the nubs?
• I still can't quite tell if the iCP keeps streaming nub coords even when you're trying to hold the nub in place (I don't think so, but it does seem very sensitive to slight movements - hence the suggestion for reducing the granularity of the coords when in this mode)
• Maybe use 2 bits within the coord char passed to specify which nub and axis the value refers too, so if the player is only moving one nub on one axis, the iCP is only sending 2 chars ('w' and value) - combined with the stream suggestions above, it might only be sending 1 char on value changes)
• I'm not sure what the iCP firmware poll rate is for detecting/sending char events - but maybe reduce it? Maybe 10hz? 5?

[MWeston]

Sometimes, letting go of all buttons and nubs, the iControlPad will continuously send a stream of 'w' (or occasionally, other constant values).
Just touching the nub will stop this stream

I believe this is part of the experimental side of this mode. When in digital controls only mode, the poll rate is 20Hz. When in combo digital/analog report mode, the poll rate can be 13 or 10 Hz. The default is 13 which is right on the edge of what the HID driver inside the BT module can handle. When you exceed its capability, it starts dropping the reports it should be sending. In your example, it looks like only the header ('w') was sent and then the rest of the packet choked so not even the KEY UP event got sent, causing a constant stream. Sending a new report (pressing a button or moving the nubs) sends a new report and if sent cleanly, clears that infinite key held down condition. The poll rate can be changed through a special command mode over SPP which is software I'm still writing slowly over time. It looks like people's creativity is starting to catch up to me and I need to get this done.

Occasionally, it appears that the iCP will drop a nub value from the 4 values passed

Quite likely related to the problem described above.

Moving a nub fully to the left seems to stop the other nub from reporting any values until you pull back a bit on the other nub.

Are you using the very latest firmware? There were some nub issues in the software that were discovered in joystick mode and it would apply here too. It was a problem related to full extent values of the nub so this is probably it.

• instead of a [-32, 32] X/Y range, a [-7, 8] range might be better - (even [-4,4] would work for what I need)
• instead of ('w' lx ly rx ry), why not just keep streaming the x/y coords without the 'w' until some other button event occurs or releasing the nubs?

I have thought of a couple ideas like this but I'm really stuck for code space now. When we ordered the microcontrollers, we were nowhere near filling the 16KB of space. With all this BT stuff and all the new features that have since gone in, I'm down to a couple hundred bytes left. I think some of the SPP stuff I have in there now can come out. One coder tried it and said it was way too laggy (manually polling controls in SPP mode) and only the auto report mode was useful.

• I still can't quite tell if the iCP keeps streaming nub coords even when you're trying to hold the nub in place (I don't think so, but it does seem very sensitive to slight movements - hence the suggestion for reducing the granularity of the coords when in this mode)

If you can keep the controls in one position then no new data gets sent. It only sends data on changes. There used to be a control register to chose between this method and constant update but constant update basically killed the communication from iCP to host (tested on my PCs and iPods). The data rate was way, way beyond what a keyboard driver expected to receive and it crapped all over itself.

There is a register that can be programmed in SPP mode that will create dead zones in the nub. You can set it to anything from 0 - 31. This can be used to prevent small, useless movements from sending a bunch of data over the link. I don't think a dead zone of 31 would be wise though.

• Maybe use 2 bits within the coord char passed to specify which nub and axis the value refers too, so if the player is only moving one nub on one axis, the iCP is only sending 2 chars ('w' and value) - combined with the stream suggestions above, it might only be sending 1 char on value changes)

That's definitely one way to do it. Right now everything is full range for people to experiment with but if no one needs that level of granularity, it could be stripped out.

I had a simple idea for something like this to go from 5 bytes down to 3: One byte per nub: <X sign><Y sign><XXX><YYY>

It gives you 7 levels in each direction with zero at any polarity meaning no deflection. With these little nubs, that might be all anyone needs and I might look into it for firmware 1.0.2.

• I'm not sure what the iCP firmware poll rate is for detecting/sending char events - but maybe reduce it? Maybe 10hz? 5?

It does appear to need to be slowed down a bit to maintain data integrity from iCP to host. Going from 13 to 10 should do it. 13 works on my unit but obviously I see now that not all devices can handle it as it is right at the edge of capability. I've been looking forward to getting feedback from others on this stuff.

[akingcoder]

Occasionally, it appears that the iCP will drop a nub value from the 4 values passed

Quite likely related to the problem described above.

Moving a nub fully to the left seems to stop the other nub from reporting any values until you pull back a bit on the other nub.

Are you using the very latest firmware? There were some nub issues in the software that were discovered in joystick mode and it would apply here too. It was a problem related to full extent values of the nub so this is probably it.

I'm currently running firmware 1.0.1a

I looked more into this issue (when pressing the nub fully to the right (not left like I said above) stopping events) and I believe I see what's going wrong. If you move either nub fully (or mostly fully) to the right, the iCP starts to only send 4 bytes instead of the normal 5 - and in the code I wrote, I assume the packet is corrupt and ignore that packet - hence no movement of any nubs on the iPhone. Here's an example (note: all tested on OSX - but also verified on iOS sim and device):

Left nub fully left

w2A@@w(B@@w$A@@w$B@@w$D@@w$B@@w$A@@w$D@@w$A@@w A@@w D@@w B@@w A@@w)A@@w@@@@

Left nub fully right

wB@@wA@@wB@@wA@@w^A@@wA@@wD@@w@@@@

Right nub fully right

w@@U?w@@]?w@@^?w@@?w@@@w@@?w@@@w@@Aw@@Ew@@@@

As you can see, the first capture is correct (as it is with all other nub directions) with a 5 byte packet, but 2 and 3 show the missing byte when either nub is fully to the right. I'm guessing this might be some sort of firmware issue - maybe overflow when at max value? This is really the only main bug I'm still seeing. I could work around it by handling a 4 byte packet - though it'd be handy to know what the dropped byte means Maybe just assume it's at max value and track which nub it most likely belongs to? BTW, anyone reading along and seeing those 'gaps' in the first capture should realize that the space character (32) is a valid value and represents -32

• I still can't quite tell if the iCP keeps streaming nub coords even when you're trying to hold the nub in place (I don't think so, but it does seem very sensitive to slight movements - hence the suggestion for reducing the granularity of the coords when in this mode)

If you can keep the controls in one position then no new data gets sent. It only sends data on changes. There used to be a control register to chose between this method and constant update but constant update basically killed the communication from iCP to host (tested on my PCs and iPods). The data rate was way, way beyond what a keyboard driver expected to receive and it crapped all over itself.

Great! - this was what I expected (only updating on position/button changes) - definitely the best way.

There is a register that can be programmed in SPP mode that will create dead zones in the nub. You can set it to anything from 0 - 31. This can be used to prevent small, useless movements from sending a bunch of data over the link. I don't think a dead zone of 31 would be wise though.

The SPP mode does look cool, but I don't think I can ask a regular iOS user to reprogram their iCP by connecting it to the PC/Mac and adjusting their deadzones Maybe defaulting to something more on the safe side is best.

• Maybe use 2 bits within the coord char passed to specify which nub and axis the value refers too, so if the player is only moving one nub on one axis, the iCP is only sending 2 chars ('w' and value) - combined with the stream suggestions above, it might only be sending 1 char on value changes)

That's definitely one way to do it. Right now everything is full range for people to experiment with but if no one needs that level of granularity, it could be stripped out.

I had a simple idea for something like this to go from 5 bytes down to 3: One byte per nub: <X sign><Y sign><XXX><YYY>

It gives you 7 levels in each direction with zero at any polarity meaning no deflection. With these little nubs, that might be all anyone needs and I might look into it for firmware 1.0.2.

Personally, I think this would be a nice way to handle it - from 5 bytes to 3. The only concern I have is whether iOS 4.* can handle an input char in the range of 0-255 instead of the normal printable ascii range of 32-127. There's a chance that any char outside of this range might get dumped before being passed to the app. Have you tried it? If you want, you can send me a firmware with this enabled and I can see what happens - and which, if any, bytes don't make it through. The other issue would is not being able to use 'w' as a reliable tag in the stream as it could be a possible nub value - though as long as it remains in 3 byte chunks, it shouldn't really be an issue - and I can still write code that can recover from the odd dropped char if needed.

• I'm not sure what the iCP firmware poll rate is for detecting/sending char events - but maybe reduce it? Maybe 10hz? 5?

It does appear to need to be slowed down a bit to maintain data integrity from iCP to host. Going from 13 to 10 should do it. 13 works on my unit but obviously I see now that not all devices can handle it as it is right at the edge of capability. I've been looking forward to getting feedback from others on this stuff.

I think 10hz would be great (I run my network physics at a 10hz input rate, so would fit in well with that ) - though honestly, I think 13 might be fine as well - assuming that bug above is related to a firmware issues instead of iOS clamping the keyboard input rate.

Thanks very much for looking into this mweston!!

[MWeston]

I'm currently running firmware 1.0.1a

Okay, well my opinion is that these problems must be related to the poll rate and the corruption that occurs inside the BT module. I posted some new firmwares you can try at different speeds just to see what happens. Maybe try the slowest (5Hz) one first and see if this still happens. The code to send the data is very basic and I don't see any logical reason for an error. Each of the four nub coordinates are added to 64 (to align it within standard ASCII) and sent to the BT module.

http://openpandora.ca/iCP_test_firmware/iCP_test_firmwares.zip

Personally, I think this would be a nice way to handle it - from 5 bytes to 3. The only concern I have is whether iOS 4.* can handle an input char in the range of 0-255 instead of the normal printable ascii range of 32-127. There's a chance that any char outside of this range might get dumped before being passed to the app. Have you tried it?

No, I haven't tried it. I forgot about the ASCII thing when I mentioned that last night. Now I understand why I had come up with this method as well:

Upper Nibble: 0X1Y (where X and Y are set to '1' if negative)
Lower Nibble: XXYY (where XX and YY are a 0-3 range for deflection)

I doubt any of this would work if you don't use standard ASCII characters so this method here was my second choice and it only has three deflection values each way. I don't know if that is enough for a general purpose driver.

[akingcoder]

I'm currently running firmware 1.0.1a

Okay, well my opinion is that these problems must be related to the poll rate and the corruption that occurs inside the BT module. I posted some new firmwares you can try at different speeds just to see what happens. Maybe try the slowest (5Hz) one first and see if this still happens. The code to send the data is very basic and I don't see any logical reason for an error. Each of the four nub coordinates are added to 64 (to align it within standard ASCII) and sent to the BT module.

http://openpandora.ca/iCP_test_firmware/iCP_test_firmwares.zip

Tried both the 5 and 10hz firmwares - unfortunately, the same result. When the nub is fully deflected to the right, it appears to drop the max value byte. I definitely notice the much slower updating at 5hz between each group of 5bytes - however, I did notice something. It looks like the 5 bytes per 'packet' are sent very quickly (looked the same rate as 13/10hz). Is it possible that there isn't enough time gap between each of the 5 bytes even at 5hz? Here's a capture with the 5hz when the left nub is deflected fully right - I bolded the bad values:

wWC@@wA@@wB@@wA@@wB@@wA@@wB@@wA@@wC@@w@@@@

Note: this was captured through textedit with the iCP connected to OSX 10.6.7 and appears to exactly mimic the output from my iPhone4 when I have the iCP connected to it. I'd be somewhat surprised if both OSX and iOS had identical rate surpression of bluetooth keyboard input - but then, maybe there's some BT standard of time between key presses?

I'm going back to the 10hz version and see if I can update the code to handle the 4 byte packets. If I track the last valid position and assume that the missing byte is a 'max' value, then I might be able to work around it.

Personally, I think this would be a nice way to handle it - from 5 bytes to 3. The only concern I have is whether iOS 4.* can handle an input char in the range of 0-255 instead of the normal printable ascii range of 32-127. There's a chance that any char outside of this range might get dumped before being passed to the app. Have you tried it?

No, I haven't tried it. I forgot about the ASCII thing when I mentioned that last night. Now I understand why I had come up with this method as well:

Upper Nibble: 0X1Y (where X and Y are set to '1' if negative)
Lower Nibble: XXYY (where XX and YY are a 0-3 range for deflection)

I doubt any of this would work if you don't use standard ASCII characters so this method here was my second choice and it only has three deflection values each way. I don't know if that is enough for a general purpose driver.

I wouldn't mind trying the full 0-255 byte packet - it's always possible that iOS will allow the values through.

[MWeston]

Is it possible that there isn't enough time gap between each of the 5 bytes even at 5hz?

It is quite possible. This special Apple mode is the only one to use this unique transfer method. If there are issues at a lowly 5Hz, then the time between reports must not be the problem. You are right about the 5 bytes being sent at the same speed for any poll rate. They are all sent one right after the other and the refresh rate is the time between the packets.

Here are some more test binaries you could try. These add a 10 character delay between each character sent to the BT module. You can try this all the way back up to the default 13.33Hz:

http://openpandora.ca/iCP_test_firmware/iCP_test_firmwares2.zip

I don't have hardware in front of me to test this at the moment but it should do what I want as long as the compiler doesn't optimize these delays out. I've had trouble with that in the past.

I wouldn't mind trying the full 0-255 byte packet - it's always possible that iOS will allow the values through.

I really don't think it will work and I actually have to code in a new method of data transfer to the BT module if I wanted to use this so I don't think the effort would be worth it. There are control signals and keys that the user never sees and I know for a fact the iOS does not pass some of them to the user. The odds that some of the characters would be lost is almost certain. Maybe at a later date I could spend the hours coding it in if you find out that there is a way to get all 256 values.

[Linopoly]

http://openpandora.ca/iCP_test_firmware/iCP_test_firmwares2.zip

I don't have hardware in front of me to test this at the moment but it should do what I want as long as the compiler doesn't optimize these delays out. I've had trouble with that in the past.

If you're using gcc, you might be able to figure out what the optimizer did by passing -S and -ggdb. Then you get an assembly file with a bunch of .loc directives that say what line the assembly code below came from. You can search that file for the line number(s) with the delay, and see if it looks like it made it into the assembly.

[MWeston]

The problem I have is creating a dummy loop to kill time like this: for (x=0;x<1000000;x++){}

The WinAVR compiler would think it was pointless and just remove it because I saw no delay at all. For tonight's code, I added a hardware bit check and conditional to the dummy loop so that the code could not optimize it out. It's not an ideal solution though. It wastes code space I can't afford but right now, it gets this test going.

[akingcoder]

Is it possible that there isn't enough time gap between each of the 5 bytes even at 5hz?

It is quite possible. This special Apple mode is the only one to use this unique transfer method. If there are issues at a lowly 5Hz, then the time between reports must not be the problem. You are right about the 5 bytes being sent at the same speed for any poll rate. They are all sent one right after the other and the refresh rate is the time between the packets.

Here are some more test binaries you could try. These add a 10 character delay between each character sent to the BT module. You can try this all the way back up to the default 13.33Hz:

http://openpandora.ca/iCP_test_firmware/iCP_test_firmwares2.zip

I don't have hardware in front of me to test this at the moment but it should do what I want as long as the compiler doesn't optimize these delays out. I've had trouble with that in the past.

Thanks for the quick firmware test binaries! I tested both the 13hz and 5hz binaries, but no luck It appears that the iCP (in this mode) refuses to send the char code 96. For both left and right nubs, when deflected fully to the right or down, they only send a 4 byte packet (includes the 'w') instead of 5 - always dropping the "`" ASCII character. Just for testing, I hooked up my normal bluetooth keyboard and pressed the ` key a few times - and it was received by the iPhone as char 96, so it's not being dumped by the OS. One other note: If I hold both nubs exactly down or to the right, only 3 bytes are sent - the two missing bytes would be the `(96) char. Any other suggestions I could try?

[MWeston]

Well thank you for all your testing.

That's a very interesting result with the lack of 96 going through. This sounds like an error in the BT module's firmware itself. I'm going to need to run some tests on hardware and not send you more binaries until I find out what's going on. Perhaps I need to send a 97 whenever a 96 was going to be sent. Hopefully that is the only error in the HID look up table, assuming that's what is wrong. I'll get back to you as soon as I can.

[QBCS]

The problem I have is creating a dummy loop to kill time like this: for (x=0;x<1000000;x++){}

The WinAVR compiler would think it was pointless and just remove it because I saw no delay at all. For tonight's code, I added a hardware bit check and conditional to the dummy loop so that the code could not optimize it out. It's not an ideal solution though. It wastes code space I can't afford but right now, it gets this test going.

In a normal compiler, declaring x as volatile should stop it from being optimized out. Of course that might be a problem if it should be optimized in other places.

[MWeston]

In a normal compiler, declaring x as volatile should stop it from being optimized out. Of course that might be a problem if it should be optimized in other places.

I thought I tried that one but maybe it was for something else. I was able to use the hardware timers for everything so far but those have a fixed period not always useful for quick delays. I may look into it again if I need these delays.

[MWeston]

It appears that the iCP (in this mode) refuses to send the char code 96. For both left and right nubs, when deflected fully to the right or down, they only send a 4 byte packet (includes the 'w') instead of 5 - always dropping the "`" ASCII character.

Well, I can confirm that the BT module sends either an invalid character or nothing when you pass a 96 to it. I modified the code to send m`w` every time you press any button and all I ever get is mw. (no period)

I modified my code to send a 97 everytime it is called upon to send a 96 and seems pretty solid, even at 13.33Hz. I moved the nubs around for a while and then parsed the big text file it made. It's tough sorting through a wall of text for 5 char blocks, but by the end of it, they all lined up so nothing was dropped anymore. I made you a couple more binaries; one at 13.33Hz and the other at 10Hz. If that first problem with the repeating w is happening, then the 10Hz should work better. I never got any issues like that but that doesn't mean they all won't. I hope this set goes better:

http://openpandora.ca/iCP_test_firmware/iCP_test_firmwares3.zip

[akingcoder]

I modified my code to send a 97 everytime it is called upon to send a 96 and seems pretty solid, even at 13.33Hz.  I moved the nubs around for a while and then parsed the big text file it made.  It's tough sorting through a wall of text for 5 char blocks, but by the end of it, they all lined up so nothing was dropped anymore.  I made you a couple more binaries; one at 13.33Hz and the other at 10Hz.  If that first problem with the repeating w is happening, then the 10Hz should work better.  I never got any issues like that but that doesn't mean they all won't.  I hope this set goes better:

Great stuff!  I tried the 13hz firmware and after updating the iControlPadTester code to convert 97->96, it now all works perfectly - even smoother than I had expected.  This is definitely usable in a game   I'll probably crank out another vid of it using the new firmware and code to show how nice it works.  Thanks again MWeston for the fast turnaround on testable firmwares.

The commit to my github repo is here

One small issue I noticed...    It looks like there's a large nub deadzone for both nubs.  You seem to have to deflected the nubs about a third of the way before there's movement.  Any chance of reducing that a bit?  If not, no big deal - I'm still happy with how much nub control we already have with non-jailbroken iPhone.  Maybe Craigx can update DizzyWiz (or some less important game app he's making) with iCP support (wouldn't even really need to publicize it - if his app sees keyboard input starting with 'm' or 'w', assume it's from an iCP.  I can't see why Apple would reject an app that supported keyboard input)

[Craigix]

I'm happy to add support if you don't mind guiding me though your method and we can try to submit it and see what happens...

[akingcoder]

I'm happy to add support if you don't mind guiding me though your method and we can try to submit it and see what happens...

Cool!  Why not first install the updated 13hz firmware above and run the iControlPadTester code from my repo to see if you find the performance acceptable?  It'd also be nice to see if someone else gets the same results I have.  It does work under both the sim and on device - though sometimes, I do have to restart the iCP when the nubs only seem to partly respond.

If you do think it'll work fine, then I can guide you through the changes - though the (non-rendering) part of the code in that repo is pretty simple.  Just add an offscreen UITextField, turn off autocorrection and autocapitalization on it, add a UITextFieldDelegate to something to receive the key events, give it focus and clear the textfield after each use.  This is all done in iControlPadReaderView class.  The ButtonStates class does the decoding and storing of nub/button events that you can then access with [ButtonStates getState:BUTTON_A], etc.  Nub values are stored packed with x in the lower 8 bits and y shifted up 8 bits.  The values are also stored offset by 32.  Here's a bit of code that retrieves the x/y value from the left nub:

            int val = [ButtonStates getState:NUB_LEFT];
            int x = (val&0xff)-64;
            int y = ((val&0xff00)>>8)-64;

x and y will then be in the range of [-32, 32].   If you need more guidance, just shout (though I need to spend most of today catching up on my Wii dev coding for work - so might not be able to answer till later).

[Craigix]

Actually, what's your opinion on faking inputs from the iCP as screen presses?

[Linopoly]

I thought I tried that one but maybe it was for something else. I was able to use the hardware timers for everything so far but those have a fixed period not always useful for quick delays. I may look into it again if I need these delays.

It looks like you can set the timer counter yourself, if you want the timer interrupt to happen early. Granted, that might complicate your code if you're using the timer for other things already.

[Stephane Hockenhull]

The problem I have is creating a dummy loop to kill time like this: for (x=0;x<1000000;x++){}

The WinAVR compiler would think it was pointless and just remove it because I saw no delay at all. For tonight's code, I added a hardware bit check and conditional to the dummy loop so that the code could not optimize it out. It's not an ideal solution though. It wastes code space I can't afford but right now, it gets this test going.

very compact delay code:

for( volatile int x = 1000000; --x;);

if that doesn't work try

{
int x;
volatile int *px = &x;
*px = 1000000;
while(--*px);
}

and if that still gets optimized out you can try:

{
volatile int x;
volatile int volatile *px = &x;
*px = 1000000;
while(--*px);
}

slightly larger code but the volatile pointer to volatile int has been enough to discourage even the most motivated (buggy) optimizing compiler.

NOTE: in all cases, 1 is the minimal no-loop delay value, NOT ZERO!
(postfix decrement/increment generates larger code than prefix, so I use prefix)

cheers

[Craigix]

Could you count up and look for the final bit being 1? <<16 or so?