Puppy Linux Discussion Forum Forum Index Puppy Linux Discussion Forum
Puppy HOME page : puppylinux.com
"THE" alternative forum : puppylinux.info
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

The time now is Wed 30 Jul 2014, 19:43
All times are UTC - 4
 Forum index » Advanced Topics » Puppy Derivatives
Sage Live - 511 -53
Moderators: Flash, JohnMurga
Post new topic   Reply to topic View previous topic :: View next topic
Page 3 of 7 [105 Posts]   Goto page: Previous 1, 2, 3, 4, 5, 6, 7 Next
Author Message
hayden


Joined: 13 Sep 2006
Posts: 59
Location: New Hampshire USA

PostPosted: Mon 27 Dec 2010, 15:03    Post subject: R in Sage  

I send this from a running CD of sagelive--511-46.iso which I downloaded yesterday. Hardware is a Dell GX-270, 1.5 Gb RAM. I know there is a Linux swap file in there but I cannot find the tools I usually use to see how big it is or whether it is in use. Lower right corner of my screen says "1.7Gb free".

Sage starts and computes 1+1;-) My main interest is in R. To run R you need to pull down a menu under the name of your Sage Notebook (top left of screen) that initially says Sage but needs to say "r" (small r!) I then did some basic stats with R. Number crunching seems to work but trying to plot data gives me an error message that X11 is not available. Since I am running Sage in X11, this seems odd. Googling around it appears I need devleopment libraries for X11, not just a functioning GUI. Here is simple R code to reproduce this:

Code:

x <- c(1,2,3)
plot(x)


I ran the script to make R available from the command line. This works but again no plots are available.

With some mesing around I was able to use the R packages already included with Sage, but attempts to install those not included gave no reposnse in Sage. Strangely, in the command-line version of R that Sage installed, I could start downloading packages. I tried Rcmdr first because it has a zillion dependencies. Looks like it may not finish today so I'll move on;-)

Let me interrupt this post to explain my interest in this project. It would be nice for professors in math/sci to hand out to students a CD with the tools they will need in their courses. In the place where I last taught this would mean R, any matrix package (such as Octave), a computer algebra system (such as Maxima) and the ability to compile C and/or Java code. Still we do not want it to get too big because student machines may not have a DVD drive or a lot of memory.

The Sage Live CD provides a lot of this. I have mixed feelings about Sage itself. The common interface is nice if you use multiple tools at once but for me it just got in the way of using R. To use regular R you just click on Console and type "R". Even a Mac user could do it;-) Sage makes you jump through a lot of hoops to get the same effect. And though you end up in a pretier terminal you still ahve to type the same text comands as in regular R. So it's a GUI only on the appearance level -- there are no menus for stats. as in Minitab or SPSS.

So, for my use I would definitely want the individual applications ready on boot up (without the student having to run a script from inside Sage). I am curious as to whether Sage takes up more or less space than jsut command-line versions or R, Octave and Maxima. (Some of those other tools are only for a very specialist minority.)

So MANY thanks to emil for his work on this. Even if the entire Sage turns out to be overkill, the process had produced all the individual pieces for us to play with!
Back to top
View user's profile Send private message 
hayden


Joined: 13 Sep 2006
Posts: 59
Location: New Hampshire USA

PostPosted: Mon 27 Dec 2010, 15:30    Post subject: PS  

It looks like the packages all downloaded but few if any installed. Probably the most common error was failure to find "make" so it looks like most of these DO require real compilation on the local machine.
Back to top
View user's profile Send private message 
emil

Joined: 10 Nov 2009
Posts: 614
Location: Austria

PostPosted: Mon 27 Dec 2010, 17:18    Post subject: about using R and other packages  

Hello Hayden,

thank you for your interest. To make the included packages available outside of sage there is the following *very* useful command:

start sage from commandline then type

Code:
install_scripts('/usr/bin/')


this writes some start scripts for the most common components like maxima or R to /usr/bin. So you can expose most of the internal packages to the user. (start R with R from terminal)

This is one of the best features of sage which I was totally ignorant for more than a year. I also think the sage developers underestimate that this could be the the real "killer" feature of the sage package.

There is no such script for python created, only for ipython (somewhat improved interface for python), but if you want to have python work from commandline e.g. make a script like

Code:
#!/bin/sh
sage -python $*


and put it in /usr/bin

The advantage is that dependecies are correct, so you can import all installed libraries (numpy, scipy, sympy).

About the installation of R-packages:
There are no compilers etc included in the base distribution of sage live. You will probably have to install the lupq_devx_511.sfs (compile environment) to install additional packages for R. To have R commander and wxmaxima working would be a nice feature! I have to admit I didn't tried to do this yet, but I am a bit short of time atm.

I would be glad if you or anybody else could try to get RCMDR or other packages working, I know this can take some time, patience and commitment so I am just asking and share a few hints:

Install the Developer package
http://boxen.math.washington.edu/home/emil/doc/html/en/ADD_SOFTWARE_devx.htm

then try to follow steps from this thread
http://www.murga-linux.com/puppy/viewtopic.php?t=51141

about the swap:

at startup a script is running, it is called swapwizard.sh (it is placed in /root/Startup).

It will allocate swapspace if needed, but it will just create a swapfile (no deticated linux swap partition). The swapfile is probably called pupswap.swp. There is an option to use a windows pagefile.sys if one is found.

more info about size of swapspace, from commandline

Code:
free


or even more detailled (filename)

Code:

cat /proc/swaps


Thanks for your interest in this project! I think most of your demands could be met by installing the Developement package, gnuplot and the octave package.

I agree with you that sage might be overkill, but it provides a coherent package of all tools, and the browser notebook interface is a good approach (setting up servers to use in a network is not difficult).

kind regards
emil
Back to top
View user's profile Send private message Visit poster's website 
hayden


Joined: 13 Sep 2006
Posts: 59
Location: New Hampshire USA

PostPosted: Mon 27 Dec 2010, 21:45    Post subject: more on R  

OK, thanks for the tip re swap space. It looks like my swap file is 2Gb and in use. I mention that only as info on what hardware configuration can run SageLive.

A separate issue is how to set up a swap for students who walk away with a CD. I think I will leave that for when all other problems are solved!-)

I did use the script to make R accessible on the command line. I think I reported data on how it worked both in Sage and from the command line.
BTW, R did eventually download packages from within Sage but this took hours with no indication anything was happening. I did mean to suggest that the iso have that script already run so the user does not have to figure that out. I remastered SageLive and the results of running the script were preserved (as was yacas -- thanks for that).

I will look at the development sfs. However, it looks like Rcmdr would not be a suitable first project!-) Rcmdr downloaded 146 files (105Mb) as dependencies! Of those, four installed and one worked. That's in resposne to a post somewhere that was hoping only a few R packages would actually require compiling;-( I think the only exceptions are packages that contain only data, say all the datasets for an R textbook.

To make a Puppy that includes R with Rcmdr seems to require an additional 100Mb for dependencies. Then if you want the user to be able to add other packages the development sfs has to be built into the iso. I am starting to think that will fill up the whole CD even without Sage or any other programs!

Then there is the issue that graphics do not work in R. I have no idea what it would take to fix that. The graphics in R are fabulous so it would be a waste to not have them. And it would make R unable to do even the tasks in an intro. stats. course.
Back to top
View user's profile Send private message 
shinobar


Joined: 28 May 2009
Posts: 2600
Location: Japan

PostPosted: Tue 28 Dec 2010, 08:37    Post subject: Re: more on R  

hayden wrote:
A separate issue is how to set up a swap for students who walk away with a CD.

If the PC has Windows, follow Menu>System>System memory>Winfonts

_________________
Google Chrome portable
Downloads for Puppy Linux http://shino.pos.to/linux/downloads.html
Back to top
View user's profile Send private message Visit poster's website 
emil

Joined: 10 Nov 2009
Posts: 614
Location: Austria

PostPosted: Wed 29 Dec 2010, 07:03    Post subject: swap space  

Hi shinobar and hayden,

swapspace:
My original indent was to have it handled "automagically".

well, there is this script (my first), swapwizard.sh Embarassed . It is called at startup (placed in /root/Startup). I originally wrote it because my first versions of sagelive were not pfix=nocopy and people had real memory problems.

It will detect if swapspace is present, and also check the available RAM. There are variables which control the limits.

Currently those are:
Code:
LOWRAM=700000     # Gives Warning if lower RAM
BIGRAM=1500000    # Don't make Swapspace if more RAM than this
MINDISK=4194304   # Only makes a Swapfile if more Space on Disk than this
MINSWAP=1048576  # Size of Swapfile


swapfiles will only be created on linux partitions by default. The Windows swapfile pagefile.sys will not be used by default. You can use it if you start swapwizard.sh with parameter -win (edit script around line 188 to change this).
There is another peculiarity, swapwizard mounts all partitions at startup, but NTFS partitions are mounted read only. If you want to write you have to unmount/remount them.

So you can adjust swapwizard to use pagefile.sys or other memory configurations, but maybe its better to handle swapspace manually and use shinobars gui for it. In that case just delete swapwizard.sh from the startup menu.

Shinobar, would it be an idea to move the swapfile settings to the first start menus? Having it under Desktop/Desktop Applets/Windows fonts is at least a bit counter intuitiv. Maybe it would fit somewhere into the gui for savefile creation?

About CD for students:
Have you ever thought to offer them an USB install - I mean it is even possible to load a bootable system on an I-pod Very Happy .

About the R plotting:
if you type r.capabilities() into the notebook it will tell that no X11 and no graphics support is present. Thats definitely bad. I also checked the official vmware image, it has the same problem.

I posted a question about the topic on http://groups.google.com/group/sage-support.
Maybe there is a reasonable fix. In the meantime It is possible to make a seperate R package which has the necessary dependencies. Since the resulting package is zipped or squashed it is possible that it still fits on CD (I guess 68 MB are free).

Actually I think the plotting difficulties are not restricted to R. Recently I tried to get TexMacs working and make a package, it worked but plotting was broken too. Do you have similar issues with maxima?

You could try to build the R package yourself, the basic R is easy to compile. The problem is to make the package with Rcmdr included, since those dependencies are installed from inside R. So it is probably necessary to make a R installation in a seperated, non default location, then install the packages you want, and at last package the whole directory tree.

If you have some days time, I would probably try it and I am especially interested if this package would also work under the new wary. But atm I am deep in making a working VM image (it's a bit tricky but I am almost there Very Happy ).

Hayeden I hope you will stay tuned and keep an eye on sagelive, I am sure there will be a solution.

Cool emil
Back to top
View user's profile Send private message Visit poster's website 
shinobar


Joined: 28 May 2009
Posts: 2600
Location: Japan

PostPosted: Wed 29 Dec 2010, 09:29    Post subject: Re: swap space  

emil wrote:
Shinobar, would it be an idea to move the swapfile settings to the first start menus? Having it under Desktop/Desktop Applets/Windows fonts is at least a bit counter intuitiv. Maybe it would fit somewhere into the gui for savefile creation?

Do you know that the SageLive automatically creates swap file, 'pupswap.swp', when the session is saved? It will be used at next boot.
This mechanism is what Barry built in recent woof.
But it doesn't work without pupsave. Barry's idea is nothing touch on HDD at Live CD mode.

Offering to create swap file at the boot up is a good idea. The user can choose touch the HDD or not.

One of my attempt is the firstrun and pupsaveconfig.
The firstrun calls the pupsaveconfig, and the pupsaveconfig offers swap file.
If you choose to make swap file, it is activated right away and in use for the session even before reboot.

There are another approaches, your swapwizard.sh, my winfonts, jpeps's swapfile manager, and etc..

_________________
Google Chrome portable
Downloads for Puppy Linux http://shino.pos.to/linux/downloads.html

Last edited by shinobar on Wed 29 Dec 2010, 18:35; edited 1 time in total
Back to top
View user's profile Send private message Visit poster's website 
emil

Joined: 10 Nov 2009
Posts: 614
Location: Austria

PostPosted: Wed 29 Dec 2010, 10:53    Post subject: swapfile  

Hi shinobar,

I think a gui supported possibility to manage and create swapspace would fit well into the first run dialogs, maybe in an advanced section so that it doesn't scare inexperienced users. It would also fit well into a Menu/System or even Menu/Filesystem.

I also second the idea that it is better not to touch harddisks without express consent of the user. I just made the suggestion of moving the gui because I felt this is the right adress Very Happy . You did enourmous work to improve and ease startup recently and also all the work with the improved localisation support which I am very grateful to have in lupq. Thanks for all this work!!
Back to top
View user's profile Send private message Visit poster's website 
emil

Joined: 10 Nov 2009
Posts: 614
Location: Austria

PostPosted: Fri 31 Dec 2010, 18:32    Post subject: Interim: R + Rcmdr available as pet
Subject description: And patch for gfortran
 

I tested the R functionality in sage and it is, well, inclomplete ...
It is possible to test enabled features from the R prompt with
Code:
capabilities()


The R which is included in the current ubuntu binaries fails in lots of those (especially no graphic support). I asked about it in the sage support forum and the answer was that it should basically work (but it doesn't), so I will investigate further with them. I think it is a bug that it doesn't work, and it is an additional bug that it doesn't work and sage -testall gives no error (not even a warning).

I will also recompile sage from source in lupq and see if I can get the missing functionality in. If I manage it there will be an upgraded version of sagelive

In the meantime I dived into it and made pets for R, and R+ R Commander (Gui). Read about it here.

However I also found a bug in sagelive's gfortran Sad ,
please install gfortran-4.4.3.pet to fix it.

Happy new year Cool
emil
Back to top
View user's profile Send private message Visit poster's website 
emil

Joined: 10 Nov 2009
Posts: 614
Location: Austria

PostPosted: Mon 10 Jan 2011, 07:23    Post subject: sagelive-511-46-r3 released
Subject description: Sage compiled from scratch, Bugfixes, working R (plotting)
 

sagelive-511-46-r3 released



Release announcment (first post of this thread)

I made a complete recompile of sage to address the fact that R is incomplete (Plotting disabled) in the official Sage binaries. Beside that there were some other bugfixes and an upgrade of gfortran.
@ hayden: problem with R plotting is solved, you can install Rcmdr in that version. It includes also shinobars new pupsaveconfig GUI(credits and thanks). Together with all the other Quickset goodies this is a "GUI only puppy".

In the short term I will work on a Virtual machine version and try to make some additional packages. In the long term I think about to make it more modular, i.e. I think about to make something like goingnuts OTF_SFS loader work and have additional sfs loaded at first boot.

Cool emil

Last edited by emil on Wed 26 Jan 2011, 17:07; edited 1 time in total
Back to top
View user's profile Send private message Visit poster's website 
emil

Joined: 10 Nov 2009
Posts: 614
Location: Austria

PostPosted: Tue 18 Jan 2011, 15:12    Post subject: NEW: sagelithe smaller version  

It always bogged me that sage is such a huge package. One reason is that it always comes as a complete developer package. With the help of the sage developers I stripped out lots stuff and created a light version of my sage pupplet., without "developer" ability, just application.

Download:
http://sage.math.washington.edu/home/emil/sagelithe

Some details:
Base distribution Lupu Quickset Edition 5.11 by shinobar

Original size of sage (uncompresssed: ca 1900 MB)
Stripped size: 770 MB (218 mB as sfs)

Size Pupplets:
SageLive - Full version, 630 MB
SageLithe - Stripped version, 400 MB
Back to top
View user's profile Send private message Visit poster's website 
nickdobrinich

Joined: 05 Apr 2007
Posts: 77
Location: Cleveland OH USA

PostPosted: Wed 19 Jan 2011, 01:10    Post subject: Sage Lithe sfs
Subject description: launch scripts
 

Emil,
This is one very terrific Puppy package.
Thank you very much for your efforts.

I have recently run the SageLive .iso versions with no problems.
I have also loaded and run the R and Rcmdr pets.

Today I d/l the smaller SageLithe sfs version and loaded it under Lighthouse Puppy 5G.
The sfs file was layered in but I do not know where to look for any kind of program launchers or terminal shell scripts.

Can you point me in the right direction?
Thx.
Back to top
View user's profile Send private message 
emil

Joined: 10 Nov 2009
Posts: 614
Location: Austria

PostPosted: Wed 19 Jan 2011, 03:10    Post subject:  

HI nickdobrinich,

The sage sfs is minimal, it contains no menu entries or shell start scripts.

Sage should be selfcontained and it has its own directory tree:
I put it under /sage.

so if you do
Code:
cd /sage
./sage                   # it will start the CLI interface
./sage -notebook         # it will start the notebook
./sage -sh              # start sage subshell
./sage -python            # it will start the the python from sage


You can also create a symlink in /usr/bin to sage, so you can call sage from everywhere:

Code:
ln -s /sage/sage /usr/bin/sage


please also note somewhere above in this thread I report how start scripts for the sage components (maxima, R, Gap, ipython ...) are generated (command install_scripts('/usr'bin/') from sage CLI).

If you prefer Menu entries you could study and eventually copy the scripts and files from your sagelive distribution.

kind regards
emil

Edit: Sage lithe is not really so minimal. e.g if you install devx package you could try
sage -sh
R
install.packages('Rcmdr', dependencies=True)
library(Rcmdr)

it should compile and install Rcmdr for you.
Back to top
View user's profile Send private message Visit poster's website 
emil

Joined: 10 Nov 2009
Posts: 614
Location: Austria

PostPosted: Wed 19 Jan 2011, 09:25    Post subject: Virtual Machine: vm-sage-lithe-a1
Subject description: Puppy Power: substantial size reduction compared to official image
 

Since Sage is just available on Linux/Mac/Solaries, but not for windows, the usual workaround is to have a copy of sage run in a virtual machine.
The sage notebook can then be started in server mode and accessed from the browser of the host machine (or from the LAN, or even from the www if the network is setup right).

Using my stripped binaries
I managed to build a virtual machine from sage (version 4.6, size zipped 407 MB, size unzipped 468 MB). This is a reduction of roughly 60% compared to the full Ubuntu based vm image currently used.


screenshot: click to enlarge

DOWNLOAD: http://sage.math.washington.edu/home/emil/sagelithe/vm-sage-lithe-a1.zip

-----
Technical Data (small vm image / current Ubuntu based sage vm image):
Virtual RAM: 512 MB / 512 MB
Virtual Disk size: 4GB / 8 GB
Virtual Swapspace: 512 MB (swapfile) / 1 GB (partition)
VMTools installed: no / yes
Jave RE: yes / yes
R - plotting: yes / no
Matplotlib-gui: yes-TclTk / no

hostname: vm-sagelithe
user: root
passwd: woofwoof (it's Puppy Linux)
admin password for sage notebook: sage

I have 4 failing doctests on my stripped binaries, but I accept this as a "user only" version, no development is intended in the VM. Also creating cython code is not possible.

It was hardest for me to get the network interface working between the virtual machine and the host system. I use the following startup script which uses a nonsecure httpl. Please comment what do you think, is this is possible in local network or is this insecure?

--
Code:
#!/bin/sh
export SAGE_BROWSER=seamonkey
IP=$(ifconfig eth0| grep 'inet addr:' | cut -d: -f2 | awk '{ print $1 }')
IPINFO=$(printf "Use http://%s to connect to the Sage Notebook Server!" "$IP")
xterm -geometry 120x9 -e sage -notebook port=80 address=$IP require_login=False open_viewer=True &
gtkdialog-splash -text "$IPINFO" -bg lightblue -fontsize large -close never -placement bottom &

----

There is still room for size reduction, because the base distro is LupQ-511/sagelive (same as live CD), I just titied up the desktop a bit (less icons). So there are still lots of applications included which are not really necessary in the vm-image (media players, flash installed, gnumerics). Some of the applications may be useful, because there are plenty of network tools (incl. samba client / server).

Also the minimum possible virtual RAM is probably lower, around 375 MB, this could be important if the vm is to run on older, "Ram challenged" hardware.

kind regards
emil

Last edited by emil on Sun 23 Jan 2011, 14:14; edited 1 time in total
Back to top
View user's profile Send private message Visit poster's website 
metamorphosis

Joined: 19 Jan 2011
Posts: 1
Location: Austria

PostPosted: Wed 19 Jan 2011, 16:45    Post subject:  

Hi Emil,

i think the nonsecure http is fine as long as the image is only used/run on single user windows. But "secure" it is not, i think. Depends how you define secure *gg*. Any program running on the host-pc could easily connect to that Webserver and do stuff. Of course normally there is no such program unless someone writes one and puts it there, so... Also i doubt setting up a plaintext http auth. would be much more secure because then a program running on the host with admin rights could just sniff the password..


Was curious and had a quick look at that latest sage-lithe image.
Don't have vmware installed at the moment but kvm, so i decided to try loading it with that.

Turns out with a little trial and error of cmdline parameters it is running fine:
Code:

qemu-kvm -enable-kvm -drive file=vmsla1-0.vmdk,if=ide -m 1024 -vga vmware -net nic,model=i82551 -net user,hostfwd=tcp::8080-:80


Now i know it's not really necessary to run sage in a VM on a linux PC, but i thought this could be useful if someone wan'ts to do some testing or development work on that image on a linux host where kvm is installed.
Back to top
View user's profile Send private message 
Display posts from previous:   Sort by:   
Page 3 of 7 [105 Posts]   Goto page: Previous 1, 2, 3, 4, 5, 6, 7 Next
Post new topic   Reply to topic View previous topic :: View next topic
 Forum index » Advanced Topics » Puppy Derivatives
Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB © 2001, 2005 phpBB Group
[ Time: 0.1302s ][ Queries: 13 (0.0050s) ][ GZIP on ]