134 replies to this topic

[Eric Jardim]

Hi all,

I know the Pandora has few NAND memory (512MB) and even fewer free NAND (~90MB). That makes the defauld package management (opkg) almost impossible to work with. I also know that it should be worked out with the PND files, that are "ready-to-use" packages that not bloat the NAND.

But I think we are loosing something with PNDs. They are cute and simple do download, but they not work as I expect. For example: when I downloaded the Gnumeric PND file, I expected that the "Open With" menu was also updated to detect Gnumeric as a possible opener of sheet files (.gnumeric, .ods, .xls, etc), but that not happened. Also, I could try typing "gnumeric" at a terminal, but it don't work. I imagine it is possible to find the path to it, but I think it is a little hidden.

I don't know if this is easy to do, as if I remove the SD card, all these features disapear. I suppose this could be done with some scripting on the PND file, but is it hard to do?

On the other hand, OPT could sell extra NAND Pandoras with 1, 2 and 4GB NAND (remember a humble Dingoo A320 have 4GB of internal flash, and it is only US$80).

I've also knew that there is a third option called Extends, that it is possible to extend the package system with the SD card, and that sounds great! But I know little about it, and it looks like an advanced feature that "not advanced" user could not benefit.

[sebt3]

But I think we are loosing something with PNDs. They are cute and simple do download, but they not work as I expect. For example: when I downloaded the Gnumeric PND file, I expected that the "Open With" menu was also updated to detect Gnumeric as a possible opener of sheet files (.gnumeric, .ods, .xls, etc), but that not happened. Also, I could try typing "gnumeric" at a terminal, but it don't work. I imagine it is possible to find the path to it, but I think it is a little hidden.

I don't know if this is easy to do, as if I remove the SD card, all these features disapear. I suppose this could be done with some scripting on the PND file, but is it hard to do?

This is a feature planned but not (yet) supported.
there is some work involved to make it happen, but it looks like the devs prefer fighting to know which have the largest d**k like here.

[Alerino]

Very good question. PND's are great except when you need to use additional or command line features. I had even installed mplayer/VLC repo version because i needed to put some specific parameters in crontab. The "Open With" is really an issue, maybe i think it's not impossible to solve it. And you can always open files from within the app. Extends also work fine.

Edit: Craig coming to bash PND and say the future is red apple bright in 3...2...1...

[mindlord]

Ask around, I'm no big fan of PND, but it does serve a specific purpose.

Don't like using PND's? There's some other options. You can install the NAND image to an SD card and opkg all you want. I do this with a 16GB card and have all the OS breathing room I could want. The process isn't hard either - easier if you have a Linux PC. Basically you extract the contents of the firmware archive onto a EXT2 formatted SD card and drop a file called boot.txt with the right stuff in it on the root of the SD card and it works

There also exists a tool that allows you to make those file associations to PND's as well. I believe it's called PQR, or something like that.

The Extend route is simpler than the SD card install route and the net result is about the same - easier if you have a Windows PC.

Increasing the NAND on the Pandora is not as easy as it sounds. The NAND is actually packaged up with the main processor along with the RAM. It's called a PoP architechure where all the parts are layered together at the Factory and replacing one part of the stack is not possible. If you really need more space the SD card install is a better (and WAY cheaper) solution.

[skeezix]

pnds have a very specific purpose; they're not perfect, but they work realyl well for msot people and for their intended goal -- apps and games that work 'like carts', nice and self contained; no install, no uninstall, creating no mess for a user; easy to copy around (not self modifying), is self documenting (documentation shows up in minimenu and in the xfce menu purely by having a pnd present), multiple apps ina single file, reflash safe, multiple SD safe, etc and so on. Great stuff ..but not perfect (nothing can be.) We could automatically create symlinks (akin to the Exec line in .desktop that points to mind) that would invpoke the app from the command line maybe..

.. but I think that most power users who want to go that far may just boot entirely from SD (near infinite space, can install using opkg or Debian or the like); you can even unpack a pnd file and it should still run just find using pnd_run or pnd_run.sh (or even cd /into/unpacked/pnd; ./runme.sh type thing.)

For command line, you can invoke them using pnd_run binary (which is also file associated to pnd so you can double click on pnds from Thunar/etc); using pnd_run by hand lets you specify which subapp within a pnd as well, in case of multiple apps.

--

Some features are not yet implemented, as sebt3 said .. we're only a few bodies, doing it in our free time (Feel free to join in and help out.. we need all the blood we can get!)

--

Something else we've not build yet which you didn't notice, is easy ways to add 'system libs' (/usr/local/lib type thing) that are mounted, but long lived, but also let you eject the SD and invalidate dependant apps; we've traditionally encouraged using the NAND based libs, or packaging other libs into the pnd so that each pnd is self contained.. but for giant libs or frameworks like java or QT or the like, its a little cumbersome.. but its not an 'easy answer' either, given our constraints (small nand, multi user handheld, etc.) We're working on it..

--

Extra NAND isn't really an option for small builds yet .. ie: making a few thousand units. When you're building 20,000 units you can sort of get away with ordering multiple classes of parts, but its not too feasible when ordering only a few thousand. You order too many of the wrong part, you're stuck with inventory issues, etc.

--

We're aiming for newbies, hackers and tinkereres, powwer users.. everyone, but sort of a happy medium in general, so the existing system is ideal for most people; a few, its suboptimal, but theres lots of options..

But if you see any big holes, just keep letting us know; we've got a request/bug tracker, and all that.

Mostly we just need more hands .. the whole firmware is built by a couple people. If you want to help out..

http://pandorawiki.o...ware_governance

--

Welcome

jeff

[Eric Jardim]

You can install the NAND image to an SD card and opkg all you want. I do this with a 16GB card and have all the OS breathing room I could want. The process isn't hard either - easier if you have a Linux PC. Basically you extract the contents of the firmware archive onto a EXT2 formatted SD card and drop a file called boot.txt with the right stuff in it on the root of the SD card and it works

Hmm, sounds interesting, but I think I am going to waste my NAND. And maybe the Pandora will be slower, as I suppose the NAND I/O is faster than SD card's.

There also exists a tool that allows you to make those file associations to PND's as well. I believe it's called PQR, or something like that.

I will look for that.

The Extend route is simpler than the SD card install route and the net result is about the same - easier if you have a Windows PC.

Fine. Where do I find easy and safe Extend instructions? I am not a basic user, but I am a bit lazy latety

Oh, I will translate your last sentence to "you will not need a Linux PC"

[skeezix]

The "stuckie extend" system is actually pretty easy, though its a tl;dr page and he's made up his own nomenclature ("prerun") which is confusing, but once you do it once, you go ooooooh, and its trivial; he's got a ready to use old debian extend you can use, and then apt-get like a madman.

http://pandorawiki.org/Extend_Utils

His extend maanger thing lets you create overlays for the union mount,and you can use them for swapfs or raw storage, or install stuff into them, etc. Pretty sweet really.

--

He's also working on PanDebian, which is using our kernel with debian debootstrap to make an actual debian port to pandora (freaky awesome.)

--

You can unpack firmware image to SD and boot from SD when needed (which can be quite fast if you get a nice class 10 SD)..

And someone is working on a "SD installer" that copies from NAND to SD to set up a bootable SD.

--

Lots of things for you to look into

jeff

[EvilDragon]

Would adding file associations a big thing?

I mean, XFCE has some standard for file associations, so basically libpnd would just need to put pnd_run with the PND into those association file and pnd_run.sh would need to support passing filenames to executables.

[skeezix]

Its not too bad no, but one more thing on the list We specced it into PXML, just not implemented into libpnd/pndnotifyd.

jeff

[Craigix]

Very good question. PND's are great except when you need to use additional or command line features. I had even installed mplayer/VLC repo version because i needed to put some specific parameters in crontab. The "Open With" is really an issue, maybe i think it's not impossible to solve it. And you can always open files from within the app. Extends also work fine.

Edit: Craig coming to bash PND and say the future is red apple bright in 3...2...1...

I don't mind PNDs actually, they work well enough, I think they are a bit over engineered, I did want to just use Zip files like Apple do, but it didn't happen.

[EvilDragon]

I don't mind PNDs actually, they work well enough, I think they are a bit over engineered, I did want to just use Zip files like Apple do, but it didn't happen.

We looked into ZIPs, it would work fine for small apps, however an issue with ZIP files is, that you can't mount them to access files, you have to uncompress them (either on RAM or SD).
Big things like Battle for Wesnoth would either take 4 minutes to uncompress on an SD Card (and you'd need the space) or it wouldn't work as they would fill the RAM

Allowing ZIPs as PNDs can be implemented though. The PND system has been created flexible, adding different compressors does work. Right now we have ISO and squashfs, but ZIP can be implemented.

Do you know how Apple solves the memory issue with big apps, if they're using zips?
Do you have a link to such zip file?

I don't have any Apple product, so I don't know much about it, but I have never heard of apple using zips, so I'd like to investigate them (to check if they also use any metadata, etc.).

[slaeshjag]

ZIP would probably not work, as zips are usually read from the end of the file, the appended PXML and Icon would probably mess it up.

EDIT: Hmm... This is weird... I've appended a bunch of garbage to a zip and unzip still handles it, I guess it might work.

[Craigix]

I don't mind PNDs actually, they work well enough, I think they are a bit over engineered, I did want to just use Zip files like Apple do, but it didn't happen.

We looked into ZIPs, it would work fine for small apps, however an issue with ZIP files is, that you can't mount them to access files, you have to uncompress them (either on RAM or SD).
Big things like Battle for Wesnoth would either take 4 minutes to uncompress on an SD Card (and you'd need the space) or it wouldn't work as they would fill the RAM

Allowing ZIPs as PNDs can be implemented though. The PND system has been created flexible, adding different compressors does work. Right now we have ISO and squashfs, but ZIP can be implemented.

Do you know how Apple solves the memory issue with big apps, if they're using zips?
Do you have a link to such zip file?

I don't have any Apple product, so I don't know much about it, but I have never heard of apple using zips, so I'd like to investigate them (to check if they also use any metadata, etc.).

I don't know how Apple handle it, but it works flawlessly.

I've used their system on apps I've made which have been almost 100MB and it is instant. Maybe it uncompresses them when you install them, I've never read up on it, but it sure makes it very easy to make packages for the iPhone (etc.)

[milkshake]

I don't know how Apple handle it, but it works flawlessly.

I've used their system on apps I've made which have been almost 100MB and it is instant. Maybe it uncompresses them when you install them, I've never read up on it, but it sure makes it very easy to make packages for the iPhone (etc.)

i have also never heard that apple use zip files, criag are u sure this is the case? if so how?

[skeezix]

Iphone stuff is normally uncompressed

Current fuse-zip and zipfs eat up all ram or we'd have used it (like a jar); we can stuff lots of formats in pnd

Btw.. Zip actually does a backward seek just like pnd.. Anything after zip footer is zip comment or 'binary junk' .. Zip will seek backwards to find the footer; sound familiar?

We though about zip long ago for that reason.. Like jar. Notive in tomcat or websphere that a war/jar/ear is unpacked at install time?

We actually thought these things through; why does everyone hink the pnd format is random crap? pnd is goofy but revolutionary

Jeffphone

[skeezix]

Also remember we have removable media and small nand.. They have 8b or more and no removal sd .. Removable and multiuser .. Pnd is pretty amazing

[WizardStan]

An IPA file is effectively a zip file with a very specific file structure. Data and binaries must go in preset directory names. It also contains an info.plist file which is basically an XML file containing all the information for the application.
That IPA file is then mounted as a filesystem. This can also be done in Linux (as well as Windows and on Mac) but it is not as flawless as Craig says. It looks flawless to the end user, but it is actually quite a bit slower than it needs to be: ZIP files were not designed with random access of files in mind, and file lookups need to be performed sequentially through the archive. They chose ZIP because it was easier on the developer, not because it was better for the end user.
A better solution would be to use squashFS, or similar compressed filesystem designed for random access. To the end user, it would have functioned basically the same, with the added bonus of file reads being slightly faster. The developers would have required a program other than some zip program, adding a small amount of additional complexity, but that's easy enough to come by, provided you have no problems with open source.

So, to summarize, Apple is using a compressed archive format (ZIP) with a very specific directory structure, containing a single XML file with basic instructions on how that application is run.
PND uses a superior compressed archive format (squashFS) with no limitations on directory structure, except that it must contain a single XML file with basic instructions on how that application is run. As a bonus, that XML is also appended to the end of the archive for even faster access: no need to scan the archive and decompress it.

Apple's IPA format is loaded as a filesystem and executes binaries within it, but to the end user looks like a single file.
Pandora's PND format is loaded as a filesystem and executes binaries within it, but to the end user looks like a single file.

tl;dr: PND is IPA with a few extra features.

[freamon]

For what it's worth, I really like PNDs! I think the fact that when you run them, all sorts of stuff becomes available to use, but when you quit your system restores to exactly it's previous state is one of it's major strengths. For something like Gnumeric, I don't think it's up to the PND author to make it accessible via "Open With ..." dialogs or the command line, since this involves changing the host system. However, if you want a PND you've downloaded to behave like it's a permanently installed application I think you could accomplish it fairly easily with a script that mounts the PND at boot and then updates your PATH and xfce.

Also, I've seen other threads mentioning 4gb NANDs. I had one in god-awful Windows CE thing I had once: it's worth remembering that whilst NAND is available in higher sizes, it's really really really slow.

[Jey123456]

http://repo.openpand...l&app=PND_utils contain a script called pnd_assoc which do let you associate pnd with a filetype. read the description for instructions.

[EvilDragon]

http://repo.openpandora.org/?page=detail&app=PND_utils contain a script called pnd_assoc which do let you associate pnd with a filetype. read the description for instructions.

Oh, this is awesome, didn't know it, thanks a lot!
With these scripts I can easily find out how associations work - and integrating that into libpnd shouldn't be too much work then