Sage Live - 511 -53

For talk and support relating specifically to Puppy derivatives
Message
Author
User avatar
hayden
Posts: 61
Joined: Thu 14 Sep 2006, 01:49
Location: New Hampshire USA

R in Sage

#31 Post by hayden »

I send this from a running CD of sagelive--511-46.iso which I downloaded yesterday. Hardware is a Dell GX-270, 1.5 Gb RAM. I know there is a Linux swap file in there but I cannot find the tools I usually use to see how big it is or whether it is in use. Lower right corner of my screen says "1.7Gb free".

Sage starts and computes 1+1;-) My main interest is in R. To run R you need to pull down a menu under the name of your Sage Notebook (top left of screen) that initially says Sage but needs to say "r" (small r!) I then did some basic stats with R. Number crunching seems to work but trying to plot data gives me an error message that X11 is not available. Since I am running Sage in X11, this seems odd. Googling around it appears I need devleopment libraries for X11, not just a functioning GUI. Here is simple R code to reproduce this:

Code: Select all

x <- c(1,2,3)
plot(x)
I ran the script to make R available from the command line. This works but again no plots are available.

With some mesing around I was able to use the R packages already included with Sage, but attempts to install those not included gave no reposnse in Sage. Strangely, in the command-line version of R that Sage installed, I could start downloading packages. I tried Rcmdr first because it has a zillion dependencies. Looks like it may not finish today so I'll move on;-)

Let me interrupt this post to explain my interest in this project. It would be nice for professors in math/sci to hand out to students a CD with the tools they will need in their courses. In the place where I last taught this would mean R, any matrix package (such as Octave), a computer algebra system (such as Maxima) and the ability to compile C and/or Java code. Still we do not want it to get too big because student machines may not have a DVD drive or a lot of memory.

The Sage Live CD provides a lot of this. I have mixed feelings about Sage itself. The common interface is nice if you use multiple tools at once but for me it just got in the way of using R. To use regular R you just click on Console and type "R". Even a Mac user could do it;-) Sage makes you jump through a lot of hoops to get the same effect. And though you end up in a pretier terminal you still ahve to type the same text comands as in regular R. So it's a GUI only on the appearance level -- there are no menus for stats. as in Minitab or SPSS.

So, for my use I would definitely want the individual applications ready on boot up (without the student having to run a script from inside Sage). I am curious as to whether Sage takes up more or less space than jsut command-line versions or R, Octave and Maxima. (Some of those other tools are only for a very specialist minority.)

So MANY thanks to emil for his work on this. Even if the entire Sage turns out to be overkill, the process had produced all the individual pieces for us to play with!

User avatar
hayden
Posts: 61
Joined: Thu 14 Sep 2006, 01:49
Location: New Hampshire USA

PS

#32 Post by hayden »

It looks like the packages all downloaded but few if any installed. Probably the most common error was failure to find "make" so it looks like most of these DO require real compilation on the local machine.

emil
Posts: 633
Joined: Tue 10 Nov 2009, 08:36
Location: Austria
Contact:

about using R and other packages

#33 Post by emil »

Hello Hayden,

thank you for your interest. To make the included packages available outside of sage there is the following *very* useful command:

start sage from commandline then type

Code: Select all

install_scripts('/usr/bin/')
this writes some start scripts for the most common components like maxima or R to /usr/bin. So you can expose most of the internal packages to the user. (start R with R from terminal)

This is one of the best features of sage which I was totally ignorant for more than a year. I also think the sage developers underestimate that this could be the the real "killer" feature of the sage package.

There is no such script for python created, only for ipython (somewhat improved interface for python), but if you want to have python work from commandline e.g. make a script like

Code: Select all

#!/bin/sh 
sage -python $*
and put it in /usr/bin

The advantage is that dependecies are correct, so you can import all installed libraries (numpy, scipy, sympy).

About the installation of R-packages:
There are no compilers etc included in the base distribution of sage live. You will probably have to install the lupq_devx_511.sfs (compile environment) to install additional packages for R. To have R commander and wxmaxima working would be a nice feature! I have to admit I didn't tried to do this yet, but I am a bit short of time atm.

I would be glad if you or anybody else could try to get RCMDR or other packages working, I know this can take some time, patience and commitment so I am just asking and share a few hints:

Install the Developer package
http://boxen.math.washington.edu/home/e ... E_devx.htm

then try to follow steps from this thread
http://www.murga-linux.com/puppy/viewtopic.php?t=51141

about the swap:

at startup a script is running, it is called swapwizard.sh (it is placed in /root/Startup).

It will allocate swapspace if needed, but it will just create a swapfile (no deticated linux swap partition). The swapfile is probably called pupswap.swp. There is an option to use a windows pagefile.sys if one is found.

more info about size of swapspace, from commandline

Code: Select all

free
or even more detailled (filename)

Code: Select all

cat /proc/swaps
Thanks for your interest in this project! I think most of your demands could be met by installing the Developement package, gnuplot and the octave package.

I agree with you that sage might be overkill, but it provides a coherent package of all tools, and the browser notebook interface is a good approach (setting up servers to use in a network is not difficult).

kind regards
emil

User avatar
hayden
Posts: 61
Joined: Thu 14 Sep 2006, 01:49
Location: New Hampshire USA

more on R

#34 Post by hayden »

OK, thanks for the tip re swap space. It looks like my swap file is 2Gb and in use. I mention that only as info on what hardware configuration can run SageLive.

A separate issue is how to set up a swap for students who walk away with a CD. I think I will leave that for when all other problems are solved!-)

I did use the script to make R accessible on the command line. I think I reported data on how it worked both in Sage and from the command line.
BTW, R did eventually download packages from within Sage but this took hours with no indication anything was happening. I did mean to suggest that the iso have that script already run so the user does not have to figure that out. I remastered SageLive and the results of running the script were preserved (as was yacas -- thanks for that).

I will look at the development sfs. However, it looks like Rcmdr would not be a suitable first project!-) Rcmdr downloaded 146 files (105Mb) as dependencies! Of those, four installed and one worked. That's in resposne to a post somewhere that was hoping only a few R packages would actually require compiling;-( I think the only exceptions are packages that contain only data, say all the datasets for an R textbook.

To make a Puppy that includes R with Rcmdr seems to require an additional 100Mb for dependencies. Then if you want the user to be able to add other packages the development sfs has to be built into the iso. I am starting to think that will fill up the whole CD even without Sage or any other programs!

Then there is the issue that graphics do not work in R. I have no idea what it would take to fix that. The graphics in R are fabulous so it would be a waste to not have them. And it would make R unable to do even the tasks in an intro. stats. course.

User avatar
shinobar
Posts: 2672
Joined: Thu 28 May 2009, 09:26
Location: Japan
Contact:

Re: more on R

#35 Post by shinobar »

hayden wrote:A separate issue is how to set up a swap for students who walk away with a CD.
If the PC has Windows, follow Menu>System>System memory>Winfonts
Downloads for Puppy Linux [url]http://shino.pos.to/linux/downloads.html[/url]

emil
Posts: 633
Joined: Tue 10 Nov 2009, 08:36
Location: Austria
Contact:

swap space

#36 Post by emil »

Hi shinobar and hayden,

swapspace:
My original indent was to have it handled "automagically".

well, there is this script (my first), swapwizard.sh :oops: . It is called at startup (placed in /root/Startup). I originally wrote it because my first versions of sagelive were not pfix=nocopy and people had real memory problems.

It will detect if swapspace is present, and also check the available RAM. There are variables which control the limits.

Currently those are:

Code: Select all

LOWRAM=700000     # Gives Warning if lower RAM
BIGRAM=1500000    # Don't make Swapspace if more RAM than this
MINDISK=4194304   # Only makes a Swapfile if more Space on Disk than this
MINSWAP=1048576  # Size of Swapfile
swapfiles will only be created on linux partitions by default. The Windows swapfile pagefile.sys will not be used by default. You can use it if you start swapwizard.sh with parameter -win (edit script around line 188 to change this).
There is another peculiarity, swapwizard mounts all partitions at startup, but NTFS partitions are mounted read only. If you want to write you have to unmount/remount them.

So you can adjust swapwizard to use pagefile.sys or other memory configurations, but maybe its better to handle swapspace manually and use shinobars gui for it. In that case just delete swapwizard.sh from the startup menu.

Shinobar, would it be an idea to move the swapfile settings to the first start menus? Having it under Desktop/Desktop Applets/Windows fonts is at least a bit counter intuitiv. Maybe it would fit somewhere into the gui for savefile creation?

About CD for students:
Have you ever thought to offer them an USB install - I mean it is even possible to load a bootable system on an I-pod :D .

About the R plotting:
if you type r.capabilities() into the notebook it will tell that no X11 and no graphics support is present. Thats definitely bad. I also checked the official vmware image, it has the same problem.

I posted a question about the topic on http://groups.google.com/group/sage-support.
Maybe there is a reasonable fix. In the meantime It is possible to make a seperate R package which has the necessary dependencies. Since the resulting package is zipped or squashed it is possible that it still fits on CD (I guess 68 MB are free).

Actually I think the plotting difficulties are not restricted to R. Recently I tried to get TexMacs working and make a package, it worked but plotting was broken too. Do you have similar issues with maxima?

You could try to build the R package yourself, the basic R is easy to compile. The problem is to make the package with Rcmdr included, since those dependencies are installed from inside R. So it is probably necessary to make a R installation in a seperated, non default location, then install the packages you want, and at last package the whole directory tree.

If you have some days time, I would probably try it and I am especially interested if this package would also work under the new wary. But atm I am deep in making a working VM image (it's a bit tricky but I am almost there :D ).

Hayeden I hope you will stay tuned and keep an eye on sagelive, I am sure there will be a solution.

8) emil

User avatar
shinobar
Posts: 2672
Joined: Thu 28 May 2009, 09:26
Location: Japan
Contact:

Re: swap space

#37 Post by shinobar »

emil wrote:Shinobar, would it be an idea to move the swapfile settings to the first start menus? Having it under Desktop/Desktop Applets/Windows fonts is at least a bit counter intuitiv. Maybe it would fit somewhere into the gui for savefile creation?
Do you know that the SageLive automatically creates swap file, 'pupswap.swp', when the session is saved? It will be used at next boot.
This mechanism is what Barry built in recent woof.
But it doesn't work without pupsave. Barry's idea is nothing touch on HDD at Live CD mode.

Offering to create swap file at the boot up is a good idea. The user can choose touch the HDD or not.

One of my attempt is the firstrun and pupsaveconfig.
The firstrun calls the pupsaveconfig, and the pupsaveconfig offers swap file.
If you choose to make swap file, it is activated right away and in use for the session even before reboot.

There are another approaches, your swapwizard.sh, my winfonts, jpeps's swapfile manager, and etc..
Last edited by shinobar on Wed 29 Dec 2010, 22:35, edited 1 time in total.
Downloads for Puppy Linux [url]http://shino.pos.to/linux/downloads.html[/url]

emil
Posts: 633
Joined: Tue 10 Nov 2009, 08:36
Location: Austria
Contact:

swapfile

#38 Post by emil »

Hi shinobar,

I think a gui supported possibility to manage and create swapspace would fit well into the first run dialogs, maybe in an advanced section so that it doesn't scare inexperienced users. It would also fit well into a Menu/System or even Menu/Filesystem.

I also second the idea that it is better not to touch harddisks without express consent of the user. I just made the suggestion of moving the gui because I felt this is the right adress :D . You did enourmous work to improve and ease startup recently and also all the work with the improved localisation support which I am very grateful to have in lupq. Thanks for all this work!!

emil
Posts: 633
Joined: Tue 10 Nov 2009, 08:36
Location: Austria
Contact:

Interim: R + Rcmdr available as pet

#39 Post by emil »

I tested the R functionality in sage and it is, well, inclomplete ...
It is possible to test enabled features from the R prompt with

Code: Select all

capabilities()
The R which is included in the current ubuntu binaries fails in lots of those (especially no graphic support). I asked about it in the sage support forum and the answer was that it should basically work (but it doesn't), so I will investigate further with them. I think it is a bug that it doesn't work, and it is an additional bug that it doesn't work and sage -testall gives no error (not even a warning).

I will also recompile sage from source in lupq and see if I can get the missing functionality in. If I manage it there will be an upgraded version of sagelive

In the meantime I dived into it and made pets for R, and R+ R Commander (Gui). Read about it here.

However I also found a bug in sagelive's gfortran :( ,
please install gfortran-4.4.3.pet to fix it.

Happy new year 8)
emil

emil
Posts: 633
Joined: Tue 10 Nov 2009, 08:36
Location: Austria
Contact:

sagelive-511-46-r3 released

#40 Post by emil »

sagelive-511-46-r3 released
Image


Release announcment (first post of this thread)

I made a complete recompile of sage to address the fact that R is incomplete (Plotting disabled) in the official Sage binaries. Beside that there were some other bugfixes and an upgrade of gfortran.
@ hayden: problem with R plotting is solved, you can install Rcmdr in that version. It includes also shinobars new pupsaveconfig GUI(credits and thanks). Together with all the other Quickset goodies this is a "GUI only puppy".

In the short term I will work on a Virtual machine version and try to make some additional packages. In the long term I think about to make it more modular, i.e. I think about to make something like goingnuts OTF_SFS loader work and have additional sfs loaded at first boot.

8) emil
Last edited by emil on Wed 26 Jan 2011, 21:07, edited 1 time in total.

emil
Posts: 633
Joined: Tue 10 Nov 2009, 08:36
Location: Austria
Contact:

NEW: sagelithe smaller version

#41 Post by emil »

It always bogged me that sage is such a huge package. One reason is that it always comes as a complete developer package. With the help of the sage developers I stripped out lots stuff and created a light version of my sage pupplet., without "developer" ability, just application.

Download:
http://sage.math.washington.edu/home/emil/sagelithe

Some details:
Base distribution Lupu Quickset Edition 5.11 by shinobar

Original size of sage (uncompresssed: ca 1900 MB)
Stripped size: 770 MB (218 mB as sfs)

Size Pupplets:
SageLive - Full version, 630 MB
SageLithe - Stripped version, 400 MB

nickdobrinich
Posts: 77
Joined: Fri 06 Apr 2007, 03:29
Location: Cleveland OH USA

Sage Lithe sfs

#42 Post by nickdobrinich »

Emil,
This is one very terrific Puppy package.
Thank you very much for your efforts.

I have recently run the SageLive .iso versions with no problems.
I have also loaded and run the R and Rcmdr pets.

Today I d/l the smaller SageLithe sfs version and loaded it under Lighthouse Puppy 5G.
The sfs file was layered in but I do not know where to look for any kind of program launchers or terminal shell scripts.

Can you point me in the right direction?
Thx.

emil
Posts: 633
Joined: Tue 10 Nov 2009, 08:36
Location: Austria
Contact:

#43 Post by emil »

HI nickdobrinich,

The sage sfs is minimal, it contains no menu entries or shell start scripts.

Sage should be selfcontained and it has its own directory tree:
I put it under /sage.

so if you do

Code: Select all

cd /sage
./sage                   # it will start the CLI interface
./sage -notebook         # it will start the notebook
./sage -sh              # start sage subshell 
./sage -python            # it will start the the python from sage


You can also create a symlink in /usr/bin to sage, so you can call sage from everywhere:

Code: Select all

ln -s /sage/sage /usr/bin/sage
please also note somewhere above in this thread I report how start scripts for the sage components (maxima, R, Gap, ipython ...) are generated (command install_scripts('/usr'bin/') from sage CLI).

If you prefer Menu entries you could study and eventually copy the scripts and files from your sagelive distribution.

kind regards
emil

Edit: Sage lithe is not really so minimal. e.g if you install devx package you could try
sage -sh
R
install.packages('Rcmdr', dependencies=True)
library(Rcmdr)

it should compile and install Rcmdr for you.

emil
Posts: 633
Joined: Tue 10 Nov 2009, 08:36
Location: Austria
Contact:

Virtual Machine: vm-sage-lithe-a1

#44 Post by emil »

Since Sage is just available on Linux/Mac/Solaries, but not for windows, the usual workaround is to have a copy of sage run in a virtual machine.
The sage notebook can then be started in server mode and accessed from the browser of the host machine (or from the LAN, or even from the www if the network is setup right).

Using my stripped binaries
I managed to build a virtual machine from sage (version 4.6, size zipped 407 MB, size unzipped 468 MB). This is a reduction of roughly 60% compared to the full Ubuntu based vm image currently used.

Image
screenshot: click to enlarge

DOWNLOAD: http://sage.math.washington.edu/home/em ... the-a1.zip

-----
Technical Data (small vm image / current Ubuntu based sage vm image):
Virtual RAM: 512 MB / 512 MB
Virtual Disk size: 4GB / 8 GB
Virtual Swapspace: 512 MB (swapfile) / 1 GB (partition)
VMTools installed: no / yes
Jave RE: yes / yes
R - plotting: yes / no
Matplotlib-gui: yes-TclTk / no

hostname: vm-sagelithe
user: root
passwd: woofwoof (it's Puppy Linux)
admin password for sage notebook: sage

I have 4 failing doctests on my stripped binaries, but I accept this as a "user only" version, no development is intended in the VM. Also creating cython code is not possible.

It was hardest for me to get the network interface working between the virtual machine and the host system. I use the following startup script which uses a nonsecure httpl. Please comment what do you think, is this is possible in local network or is this insecure?

--

Code: Select all

#!/bin/sh
export SAGE_BROWSER=seamonkey
IP=$(ifconfig eth0| grep 'inet addr:' | cut -d: -f2 | awk '{ print $1 }')
IPINFO=$(printf "Use http://%s to connect to the Sage Notebook Server!" "$IP")
xterm -geometry 120x9 -e sage -notebook port=80 address=$IP require_login=False open_viewer=True &
gtkdialog-splash -text "$IPINFO" -bg lightblue -fontsize large -close never -placement bottom &
----

There is still room for size reduction, because the base distro is LupQ-511/sagelive (same as live CD), I just titied up the desktop a bit (less icons). So there are still lots of applications included which are not really necessary in the vm-image (media players, flash installed, gnumerics). Some of the applications may be useful, because there are plenty of network tools (incl. samba client / server).

Also the minimum possible virtual RAM is probably lower, around 375 MB, this could be important if the vm is to run on older, "Ram challenged" hardware.

kind regards
emil
Last edited by emil on Sun 23 Jan 2011, 18:14, edited 1 time in total.

metamorphosis
Posts: 1
Joined: Wed 19 Jan 2011, 20:07
Location: Austria

#45 Post by metamorphosis »

Hi Emil,

i think the nonsecure http is fine as long as the image is only used/run on single user windows. But "secure" it is not, i think. Depends how you define secure *gg*. Any program running on the host-pc could easily connect to that Webserver and do stuff. Of course normally there is no such program unless someone writes one and puts it there, so... Also i doubt setting up a plaintext http auth. would be much more secure because then a program running on the host with admin rights could just sniff the password..


Was curious and had a quick look at that latest sage-lithe image.
Don't have vmware installed at the moment but kvm, so i decided to try loading it with that.

Turns out with a little trial and error of cmdline parameters it is running fine:

Code: Select all

qemu-kvm -enable-kvm -drive file=vmsla1-0.vmdk,if=ide -m 1024 -vga vmware -net nic,model=i82551 -net user,hostfwd=tcp::8080-:80
Now i know it's not really necessary to run sage in a VM on a linux PC, but i thought this could be useful if someone wan'ts to do some testing or development work on that image on a linux host where kvm is installed.

emil
Posts: 633
Joined: Tue 10 Nov 2009, 08:36
Location: Austria
Contact:

#46 Post by emil »

Servus methamorphosis :D ,

I have no network experience, and I also think as long as there are only trusted users in the LAN it is fine. To see the VM from "outside, like www" I think it have to be routed somehow (I really know nothing about networks).

The problem could be, if you log in to the Sage Notebook you have a full programming environment, running currently as root, so you could (maybe?) do all sort of stuff there. I know there is the possibility to run applications in puppy as different user (like kirk does in Fatdog, browser runs as user spot).
This could be a possibility to restrict damage, but currently I just have a vague idea about all possible implications!

So thanks for kind feedback - another question. You are running qemu with kvm - how is the speed compared to vmware or Virtual box?

Thanks for the startup-command for quemu - this line definitly qualifies for higher arcane commandline magic!

8) emil

HAHA: 1.post - welcome to puppy forum!

User avatar
Aitch
Posts: 6518
Joined: Wed 04 Apr 2007, 15:57
Location: Chatham, Kent, UK

#47 Post by Aitch »

Hi emil....

I don't know if it's connected to your SageLive, but user L18L found sage-4.6-32bit-Puppy_Linux_Lupu.sfs at

http://www.mirrorservice.org/sites/www. ... index.html

In response to my post about [dismal] Puppy mirroring

http://www.murga-linux.com/puppy/viewtopic.php?t=64859

It may be for Lupu users only, I don't know?

Aitch :)

emil
Posts: 633
Joined: Tue 10 Nov 2009, 08:36
Location: Austria
Contact:

Sage for puppy on the official Sage servers

#48 Post by emil »

Hi Aitch,
this is a Sage package compiled in November. It was compiled at 5.11. For other puppies: Sage is usually compiled from source. One needs the devx and gfortran installed.
I am sorry to say that, but given the fast release cycle of sage and the multitude of different puppies, it I find it next to impossible to support up-to-date packages for a broader range of puppies.

In the meantime there is already a new release (4.6.1). For 511 users, who dont need developement capabilities I recommend the "sagelithe" package which is only 218 MB (unofficial).

Emil

emil
Posts: 633
Joined: Tue 10 Nov 2009, 08:36
Location: Austria
Contact:

Purity

#49 Post by emil »

Image
from http://xkcd.com/435/

couldn't do any real work on sagelive derivative latly. Maybe there will be an update to a new sage version in autumn.

ICPUG
Posts: 1308
Joined: Mon 25 Jul 2005, 00:09
Location: UK

#50 Post by ICPUG »

Hi Emil,

That's funny - and probably confirms the mathematics stereotype to the rest of the world!

Now where does engineering appear on this scale?

You update SageLive when you are able - less frequent updates can be an advantage sometimes!

I was playing with the Knoppix 6.5 DVD yesterday and noticed a maths package in the education section of the disk. I would tell you what it was called if I was on my own machine now! It was a gui based package which I quickly got to do a simple indefinite integration (x^3). Trying to find the name of the package just now I find there is a www.knoppix-math.org website that maybe relates to a Knoppix disk for mathematicians. Don't know cos my work's internet policy will not let me see the website! Lookin at the Google cache shows me it is mainly a Japanese project although an English version has recently been released.

Take care.

Post Reply