Gentoo Optimization on PS3

[asong26]

Hi,

Wanted to kickoff this thread, as I feel that it's my turn to learn from Hax. So.., Mr. Hax, please keep us updated while your PS3 is compiling away as to how you've you are customizing your own fresh install of Gentoo without relying on the stage4 tarball. We would loooove to learn!

BTW:

You can check the build flags on the -G option. Use the -pvG option. The USE flags will show. The USE flag that return after this command will represent how that binary was compiled (which USE flags were used).

Conversely, if you use -pv only, you will see how that app will be installed based on default options +, -, your make file configuration. So, -pvG will always show how that binary is installed regardless of your make file settings.

Keep us posted on your progress. As I firmly believe that this is the real way of doing it. Afterall, the stage4 tarball that we use are compiled using PowerMac G5 as confirmed by the creator (rangerb) of these tarballs. I'd be real curious to see how well it performs.

You're taking a great step for all of us. Not to say this hasn't done before, but I don't think anyone has done this recently, leveraging the latest toolchains. I think people like tokka over in Japan did this first, but back then the toolchains were very immature.

[haxen]

Hey asong, thanks for the vine ...

A real nice simple command to see all storage devices available during the install process is fdisk -l
The -l will give you a listing of devices and won't reset the file allocation table. An example of my PS3 as it sits now:

localhost ~ # fdisk -l

Disk /dev/sda: 47.1 GB, 47126736896 bytes
64 heads, 32 sectors/track, 44943 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes

Device Boot Start End Blocks Id System
/dev/sda1 1 43919 44973040 83 Linux
/dev/sda2 43920 44943 1048576 82 Linux swap / Solaris

Disk /dev/sdb: 4 MB, 4194304 bytes
1 heads, 8 sectors/track, 1024 cylinders
Units = cylinders of 8 * 512 = 4096 bytes

Disk /dev/sdb doesn't contain a valid partition table

Disk /dev/sdf: 1048 MB, 1048481280 bytes
33 heads, 63 sectors/track, 985 cylinders
Units = cylinders of 2079 * 512 = 1064448 bytes

Device Boot Start End Blocks Id System
/dev/sdf1 * 1 985 1023874+ 6 FAT16
Partition 1 has different physical/logical endings:
phys=(984, 32, 63) logical=(984, 32, 56)

This is where you discover what name Linux assigns to your USB storage device. Look above and you see the /dev/sdf1 device showing a Gig of memory, that is my digital camera connected by USB to PS3.

During the install process I go

cd /mnt
mkdir camera
mount /dev/sdf1 /mnt/camera/

And now I can transfer large tarballs, portage overlays, kernels etc. from the camera memory (copied files to camera from PC) to PS3 during install process. I am not just stuck with the DVD iso files and don't need to wait for PS3 to link to internet sources during install. This means I can get aggressive durring install and load up whatever pretty quickly and If I don't like the results, I just reinstall without much time lost.

So I compiled the 2.6.16 kernel and grabbed some of the stage4 tarball and of course the latest portage overlays and then started to build the system using the Gentoo on PS3 wiki How To as a guide. I wanted KDE as a WM for eye candy and if I build it lean, then maybe I don't need to go to XFCE for quickness.

So I have since compiled libs, PyQt, imagemagick, kdebase, kicker, superkaramba, mplayer (from wiki), aaq to play mp4s, liquid weather plug-in for superkaramba and I now have a very lean KDE. Of course I don't have a lot of the nice and somewhat repetitive or seldom used apps that normal KDE has, just what I want which is terminal, konqueror file browser in superuser mode, configurations, kicker.

I have done this by using for instance

emerge --search kde

and looking at what various kde packages and components are out there and hand picking some basic ones along with kdebase

Then i emerge what I want with for instance

emerge -va imagemagick

The v flag above gives a verbose listing of dependencies and use flag information about what is going to be installed and you can glean good info about what you need to run a particular app that you searched for. the a flag halts the emerge and asks if you want to proceed which I recommend everyone use.

This gives you a chance to quit if you see too many dependencies and you dont have the time to wait, plus it gives you a chance to cut and paste the recommended use flags into your make.conf file which I recommend that you have loaded up and ready in a vim editor session (or nano) in a second terminal window during this compile process. Just update the use flags in make.conf as per recommendation from the verbose listing, then save make.conf (F1 and then :wa in vim), then back to compile window and hit enter (yes) to install & compile.

the emerge -G or -g is a nice idea to save on compile time, but I don't necessarily trust what someone has packaged up and put up on the package sites, it might be a little bloated so I compile my own stuff so far.

Now what I still need to do is:
- bittorrent
- good ftp program
- gimp
- firefox
- java for firefox (see thread in this forum)
- gaim
- instant messenger (need to ask guy who posted latest pic in ps3 pics about what is name of that messenger app on his desktop)
- vlc
- mythtv

I do believe I am building a lighter weight Gentoo with eye candy but hand picked and compiled apps, trying to save memory and loading on the under-resourced PS3.

Of course playing with the make.conf flags like -jv3 vs. -jv2 and different mcpu types etc. is kinda neat as to how it affects compile time etc.

I haven't spent much time with xorg.conf but was going to really try to remove the black border of my plasma display in 720p when I found that I could overscan the plasma display to fit the whole screen now !

[asong26]

Good stuff. One thing I am looking forward to is an update toolchain that allows you to enable more SPUs, without the memory limitation issues during compile time. If I understand correctly there memory issues to be consider for compiling these apps with more SPUs enabled, as well as running these apps after they've been compiled...

After you do all that, you should backup your system as a disk image, and then see how much memory you're actually using at normal usage environment by typing: free

I'd like to see some delta of the memory usage.

Anyway, all good stuff. I can't wait to hear your results.

[ksoh624]

Very impressive. I was just about to move over to Gentoo from YDL.
Should I wait for your results or should I go ahead and change over anyhow?

[asong26]

I'd say change over now. Get used to Gentoo, using the official tarball install. When you feel comfortable, you would have a lot easier time experimenting with Gentoo...

[ksoh624]

Oh, its not that I'm not comfortable using Linux just wasn't sure of the time frame till Hax releases something. But ya I'll change over anywho.
Thanks

[asong26]

Yo Hax!

So, what's the verdict? Should we all recompile our own kernel and rebuild everything from ground up? Did you notice any changes in performance? Somehow I doubt it....

Prove me wrong.

[haxen]

Hey whatsup guys, Asong26... Props to you guys for getting it done on the site here.
So after 4 builds of Gentoo, I think my current setup is squeezing the most that I can get out of the PS3 so far, but it is a long process.

Gentoo on the PS3 = super slow compiling, but the goal is to have just what I want rather than loading up prebuilt packages. I would caution that it takes perseverance and some Linux knowledge before I would recommend anyone else trying to custom compile their setup. Asosng could easiily do it, but others who are just getting used to Linux and don't want to trial and error setups and don't delve into emerge without the -G or -g, and -umerge and --depclean etc. on installations (re-installations in some cases) but want windows-type software installations, then stay away from this.

What are the results?
1. In startup mode in KDE, am I running with less RAM usage than say, Asong26? Tough question, depends on my apps and services in memory compared to Asong26. But I am willing to bet that my KDE base system takes up less memory (physical and logical), but the kernel is probably exactly the same.

2. Do I have all standard WM apps installed? no, just apps that I want. My WM (KDE in my case) apps are very few, by design. Example? My KDE menu for System apps has 3, just 3 apps in the menu. How many does yours have?

3. We should compare apps and services in memory after startup. Remove all user selected apps from startup and lets compare tables and RAM. Lets use KDE as the base WM. I won't custom hack out any KDE basic services, just remove superkaramba plugins and Amarok from my startup. I will post my table here later and Asong26 will put up his.

More later.

Oh, one thing that I have developed over time is a complete migration to using the package use keywords approach to compiling and updating versus the make.conf use flags. The concept of global use flags versus app-specific use flags is a very important one that I learned the hard way. I believe this concept is the most important thing that I have learned so far in Gentoo. Maybe we should try to develop here a thread of the best make.conf and the best package.use files for the PS3.

[asong26]

I am pretty content with my current setup (except for the lack of commercial DVD playback), but I would suggest that you make a backup copy of your install for general backup as well as test bed. You can either tar up a stage4 tarball, or you can use an imaging software for compressed drive image. It's called 'partimage' Once the image is created, you can wipe out the partition and install the official stage4 tarball, and see what memory usage is with just the kdebase installed. Once that's done, you can restore your image, and do another test to see if there's any significant differences.

In my environment I have KDE, GNOME and XFCE4 installed. I've compiled XFCE4 with KDE and GNOME integration, so I am able to use this lightweight WM while having the accessibility to either KDE or GNOME app without bogging down the memory when they're not in use. If you haven't done so already, you should give XFCE4 a try. It very fast, and a lot smoother in my opinion, regardless if I am using KDE / GNOME applications with it or not. One of the first things you'll noticed is the mouse movement, and tighter the controls are. Second, the app load times seem to be faster vs. kdebase.

Just out of curiosity, how long did it take you to compile the kernel and the necessary systools to have it up and running?

[asong26]

BTW:

This is the memory snapshot on above setup, running XFCE-4 with only the kkonsole (kde terminial console) running....

total used free shared buffers cached
Mem: 201136 116896 84240 0 732 52868
-/+ buffers/cache: 63296 137840
Swap: 965624 2948 962676
asong@localhost ~ $


What does your memory look like when you've got nothing running except for the command prompt? You should do a comparison of memory usage with or without your current themes w/ widgets on the desktop. Restore your desktop with no back drop, using a simple color and don't open any apps. Let me know...

I'd be really surprised if you could beat these numbers using KDE as your WM....