YesI guess I'm going to have to learn genie.
Let was first added in the Korn shell, and Bash got it from there. Performance improvements were a focus of ksh, and one solution was built-ins to replace things like eval that had to be called as external commands. Replacing /bin/echo with a built-in (that is an alias to the print primitive) was another example.Pizzasgood wrote:One way of processing files with Bash is with the read command, by piping the contents through 'while' with 'read LINE' as the test:I wasn't aware of the 'let' command. I've been using eval for things that need to be ash-compatible, and $[$VAR+$i] for Bash-only things. The square brackets are much faster than eval in Bash and slightly faster than let, but are not supported in ash. The 'let' is many many times faster than 'expr', and ash using let is significantly faster than Bash with any of the above. And the speeds of let and square-brackets are even closer when using #!/bin/sh rather than #!/bin/bash. So I'm going to be using let from now on. Good stuff. Thanks.Code: Select all
cat /some/file.txt | while read LINE; do echo "The following line is one line of text from /some/file.txt:" echo $LINE done
Python is an interpreted language powerful enough that you can write complete applications in it, including fancy GUI stuff using PythonTK. It's also cross-platform, and available for Windows, Mac OS/X, and *nix. (I have several text editors in my collection written entirely in Python, and a couple of others that use Python as an extension language. Eric S. Raymond was going on at one point about what Lisp got wrong and Python got right. I suggested he rewrite Gnu Emacs, which is based on Lisp, to use Python instead, and he said "Don't tempt me!"They all have a loop or two. But the actual tasks are mostly using generic programs like 'cp', 'tar', 'mencoder', etc. What benefit would there be in using Python for that? It seems to me like doing that stuff in anything else would just be inserting an extra layer of complexity.
I'm not trying to be argumentative. I've actually been very tempted to use things besides Bash on some projects, particularly things that need arrays. But for the above examples, it seems to me like Bash is ideal for them.
)The usual solution to that is to use Perl, which is intended for just such cases, and replaces sed and awk.Now, in a script I recently wrote that does some weird interactions with the output of some 'git' commands, maybe Python would have been easier than all the calls to 'grep' and 'sed' and futile attempts to get Bash's string and array parsing to work the way I needed.
Genie bears some syntactic resemblance to Python, so I suppose you might be able to do it, but I'm not sure why you should. The advantage you would get is that Genie "compiles" to C, which is then compiled to native machine code, so your executable will theoretically be faster. But if that's a concern, you are better not writing in Python in the first place, or translating Python code to C without the intermediate Genie step.)Lobster wrote:YesI guess I'm going to have to learn genie.
http://puppylinux.com/genie/
You could even create a python to genie parser
- improving your understanding of both languages
Even I am having a go . . .
http://www.murga-linux.com/puppy/viewto ... 314#283314
Barry and MU have a record of producing Puppy code
Vala is being used by Gnome
Linus has recently moved to using Gnome
So the future is set already
Can you do a parser?
If you can deal with Ruby, you can deal with Python. Ruby was developed to address perceived lacks in Python.Lobster wrote:Yep if speed is no problem the future would be the slowly
but easy to use Ruby.
You will be better served to consider just what problem you are trying to solve. There is a reason why so many languages exist, and there isn't a "one size fits all" solution.My approach is what language is being used by Puppy developers
or being explored by them?
What they write is our future.
Not me. I have too many other projects either planned or underway as it is.1. Who is willing to help me?
If you want to replace Bash, it needs to be able to handle the situations I showed at least as elegantly as Bash did. And if you want to completely replace Bash, to the point where it isn't even included, you'll have to either write a bash (or at least sh) emulator, or have a wrapper named "bash" set up to run the script through a bash->python translator and then run the translated script with Python. Gotta support that occasional bash script you'll get from random 3rd party packages.2. What features does this Python shell need?
GTK 23. What is the prevailing GUI system for puppy?
I don't even want to think about any kind of BASH translatorPizzasgood wrote:If you want to replace Bash, it needs to be able to handle the situations I showed at least as elegantly as Bash did. And if you want to completely replace Bash, to the point where it isn't even included, you'll have to either write a bash (or at least sh) emulator, or have a wrapper named "bash" set up to run the script through a bash->python translator and then run the translated script with Python. Gotta support that occasional bash script you'll get from random 3rd party packages.
. (the syntax sucks) Getting Python to the stage were BASH is not needed is a long way off anyway so there's some time still. But, would someone make a list of features a good shell needs? I really have no idea so if someone could either link a list or write one up that would be wonderfull. Jamie MccrackenMr. Maxwell wrote: 4. How can I contact the guy who made Genie?
Maybe you should get some experience with shell scripting first. I mean, if you want to replace something, you kinda need to know what you're replacing in great enough detail to do a good job. Know thine enemy, they say.But, would someone make a list of features a good shell needs? I really have no idea so if someone could either link a list or write one up that would be wonderfull.
This is odd but absolutely true. Python "should" be easier to learn (and probably is) but bash (or Bourne) is so endemic to UNIX/Linux programming that anyone and everyone wanting to play with Linux system operation pushes themselves to learn the sometimes ungainly and painful bash syntax. I did, and I also learned C (for other reasons though). Later I worked hard to learn Python, and though it is an extremely easy to understand and read syntax, I nevertheless struggled terribly to get on top of it - as MU suggests earlier, for some bizarre reason my brain had become too used to the less than nice looking syntax of C and the way it almost exactly models the way data is used in UNIX. But that brings me to the following, which I don't agree with at all [EDIT: sorry... I do agree with DMcCunney to some extent afterall; see my next post]DMcCunney wrote: Whether it's more human readable is entirely a matter of whether you know the language. A lot of folks are proficient in Bourne shell scripting (and the bash script language is Bourne shell with extensions.) Less people are proficient in Python and TclTk.
Linux, as with modern UNIX, is written in C, though in some ways the relationship between C and UNIX is like the chicken and the egg scenario. It is as if C is written 'for' UNIX/Linux; it provides all the system calls and functionality to provide pipes and redirection. In many ways, all the bash shell does is provide a slightly more convenient way to use the underlying system calls - via libraries of very simple C routines. Though an interpretive language, each utility provided by bash is actually very efficient (since small bits of C code). But C doesn't store data as ASCII strings at all (in fact C doesn't 'really' have a concept of 'ASCII string' handling at all), everything is stored as binary with an address pointer marking the beginning and a null character (a binary byte) marking the end of the so-called 'string' (a binary string). i.e. its stored as binary, not ASCII. Pipes handle binary data, not just ASCII and the facility is actually provided by C system calls; bash just provides a human user interface to the facility.DMcCunney wrote: First, Unix was developed by programmers, who wanted a better environment for software development. Software source code is all ASCII, and the implicit assumption of the shell and pipelines is that programs are passing around ASCII strings. If the data you are diddling is not ASCII, this breaks down.
Compared to what? The base syntax is System V Bourne shell, with some ksh and csh stuff tossed in. It's not that crufty.Mr. Maxwell wrote:I don't even want to think about any kind of BASH translator
Don't even try.Getting Python to the stage were BASH is not needed is a long way off anyway so there's some time still.
It's not just bash (or the Bourne shell, C shell, Korn shell, Z shell, etc.). The various utilities you call from shell scripts also tend to assume ASCII strings. Awk, diff, grep, sed, tr and the like are intended to process ASCII files. As mentioned, the developers of Unix were software developers, and many of the utilities provided with a *nix system are intended to manipulate and transform source code in various ways.mcewanw wrote:On further reflection, and to be fair to DMcCunney, it is true that though the pipe mechanism provided by C system calls themselves handle binary data and not just ASCII the slightly higher level interface provided by bash, for example, basically expects ASCII strings (which can be a real nuisance/drawback at times).
Precisely. If what you are dealing with is binary data, a shell script may be the wrong approach to take.The answer to that is generally to rewrite the whole bash script in C with calls to the std libs (which is often a relatively easy thing to do) or to simply interface the main bash script with a small C utility, which accomplishes the lower level trickery that the more general purpose bash cannot.
Very true in general, and in traditional UNIX "philosophy" - bash, Bourne, Korn and so on fitting in with that philosophy. Just didn't want anyone imagining that ASCII was an enforced limitation. There are many utilities which also work with binary strings and such utilities can easily be written in C for calling from bash. The command cmp for example compares files byte by byte; the files don't need to be ASCII files as far as I remember. But the traditional shell, along with utilities such as those you mentioned, was designed and optimised for processing text files, which is a very transportable format (which, via the likes of the more recently thought up XML way of formatting text data can also be provided with a complex structure yet remain able to be processed/parsed/transformed with the old UNIX utils such as sed and awk). Awk's a particularly interesting utility too, of course, since it is a programming language in its own right.DMcCunney wrote: It's not just bash (or the Bourne shell, C shell, Korn shell, Z shell, etc.). The various utilities you call from shell scripts also tend to assume ASCII strings. Awk, diff, grep, sed, tr and the like are intended to process ASCII files. As mentioned, the developers of Unix were software developers, and many of the utilities provided with a *nix system are intended to manipulate and transform source code in various ways.
You pretty much have to to advance beyond the end user level, since most of the system is configured and controlled by shell scripts.mcewanw wrote:This is odd but absolutely true. Python "should" be easier to learn (and probably is) but bash (or Bourne) is so endemic to UNIX/Linux programming that anyone and everyone wanting to play with Linux system operation pushes themselves to learn the sometimes ungainly and painful bash syntax.DMcCunney wrote: Whether it's more human readable is entirely a matter of whether you know the language. A lot of folks are proficient in Bourne shell scripting (and the bash script language is Bourne shell with extensions.) Less people are proficient in Python and TclTk.
It's not a surprise. The development of computer languages has involved progressively higher levels of abstraction. If you are accustomed to and comfortable with dealing with the level provided by C/C++, Python can be a challenge. The data you are processing just doesn't look the same.I did, and I also learned C (for other reasons though). Later I worked hard to learn Python, and though it is an extremely easy to understand and read syntax, I nevertheless struggled terribly to get on top of it - as MU suggests earlier, for some bizarre reason my brain had become too used to the less than nice looking syntax of C and the way it almost exactly models the way data is used in UNIX. But that brings me to the following, which I don't agree with at all [EDIT: sorry... I do agree with DMcCunney to some extent afterall; see my next post]
Well, more likely C++ now, which is not the same thing. (A lot of issues stem from programmers trying to treat C++ like it was C.)Linux, as with modern UNIX, is written in C,DMcCunney wrote: First, Unix was developed by programmers, who wanted a better environment for software development. Software source code is all ASCII, and the implicit assumption of the shell and pipelines is that programs are passing around ASCII strings. If the data you are diddling is not ASCII, this breaks down.
Up through about v6, Unix was written in Macro-11, the assembly language provided on the DEC mini it was developed on. Around v7, C had matured enough to be substituted, and most of Unix was rewritten in it. The design of Unix influenced C, and vice versa.though in some ways the relationship between C and UNIX is like the chicken and the egg scenario. It is as if C is written 'for' UNIX/Linux; it provides all the system calls and functionality to provide pipes and redirection.
We may be talking about different things. From my viewpoint, bash doesn't provide the utilities. Bash implements high-level control constructs you can use to tie external utilities together to perform processing tasks. The Unix philosophy was one tool for one job, each optimized to do a particular thing. If you needed to do more than one thing to data, you tied together more than one tool in a script.In many ways, all the bash shell does is provide a slightly more convenient way to use the underlying system calls - via libraries of very simple C routines. Though an interpretive language, each utility provided by bash is actually very efficient (since small bits of C code).
Agreed. C is a language intended for systems programming, and can manipulate any kind of data. I never suggested C used or expected ASCII. I said the sort of things you used shell scripts to do expected ASCII.But C doesn't store data as ASCII strings at all (in fact C doesn't 'really' have a concept of 'ASCII string' handling at all), everything is stored as binary with an address pointer marking the beginning and a null character (a binary byte) marking the end of the so-called 'string' (a binary string). i.e. its stored as binary, not ASCII. Pipes handle binary data, not just ASCII and the facility is actually provided by C system calls; bash just provides a human user interface to the facility.
Agreed. And why bolt it on to python? A lot of what was done by shell scripts calling external utilities is now done in perl, which includes the capabilities formerly provided by things like awk and sed.So why look for a C extension module to bolt onto Python in order to get rid of bash (or Bourne, Korn or the C shell etc)? Bash IS an optimised extension module written in C for just the purpose required. But the syntax is ugly, because it is so low-level and close to C itself. bash just mirrors the underlying C system calls and, in a way, therefore, UNIX/Linux mirrors C. In human culture the individual, some say, is constructed from language - we 'are' the language that produces us. The relationship between C, bash, and Linux, follows that same model.
"One language to rule them all, one language to bind them..."But I absolutely agree that much of the time it would be nice to write in a more human-oriented computer language - for much of ALL programming, including scripting. And why have different syntax for that higher level scripting language in comparison with the compiling language for producing faster executables (where speed of execution is critical)?'
You can actually get the low-level calls in Python. It's a language in which complete applications can be written, and as such requires that sort of access. It simply provides a higher level of abstraction.So I also agree thoroughly with the concept being described by BigPilot at the start of this thread and argued strongly for by Mr. Maxwell. Python syntax for both the main system script language AND the compiling language (e.g. Genie) is a holy grail to look for. But you still need bash (as the lower-level C extension module/library of routines) to provide that level of system call interface to the higher level Python or Python-like code. No need to reinvent the wheel - you still need a pretty low level library of system calls such as provided by bash.
As far as I can tell, the problem isn't that Python is big. (The PET for Python 2.5.4 is 4.5MB.) The bigger problem is that Python is really intended for faster, more powerful hardware than the sort of machine Puppy tends to be installed on. Barry complained in a blog post he though Python was bloated. Well, yeah, if your target system is a P200 with 64MB of RAM...The only problem is that Python itself tends to be too big to contemplate using as a scripting language for a Linux distribution which is purposively being kept unbloated in size.
Why?Lua would be a better fit. But then we need a version of say Genie that also provides a Lua-like syntax.
There is an effort to add gtk to Lua 5. See http://oertl.at/wolfgang/en/project/lua-gtkAnd the scripting language Lua needs to provide a GUI toolkit that looks good and is compatible with the most popular applications - i.e. a version of Lua, say, that interfaces to a gtk2 toolkit. John Murga's murgaLua uses FLTK, and that is probably its downfall; otherwise I'm sure such a tiny, powerful language would have been a permanent part of all Puppy distributions.
I think Genie/Vala were intended precisely to address that "difficult to learn" issue, but they still use C under the hood.Useful though gtkdialog has been to the Puppy community, it is limited in power (and that limitation has a major effect on Puppy development in my opinion); Genie/Vala looks like a very positive way forward (but still I'd like a scripting language with the same or similar syntax (with bash as an extension to provide the lower-level pipe interface and so on). Because of the size factor, I still favour a Lua syntax, especially since Lua, like Python is useful as a cross-platform language and a very popular language, actively used by the gaming community for scripting purposes. If only Genie/Vala was more popular/cross-platform, maybe C and C++ would become less important (which would make a lot of people happy since they are hardly considered easy to learn, like alone the associated "popular" GUI toolkit interfaces for them).
It's also a lot smaller and less resource intensive, and is a good for for low-end oriented distros like Puppy. DSL makes extensive use of it.Lua isn't 'quite' as pretty or obvious as Python to look at, so I honestly believe that I have a much better chance of mastering it.
Why... because nice though Python code looks I remember being bored stiff with the language (no idea why). But ... I dabbled much more recently with Lua and that interested me. And ... I'm lazy. I would also sometimes like to use a scripting language which had less of the idiosyncrasies endemic to bash; getting a bash script to work as planned can sometimes be a real pain to the brain. But I would also like to use a higher level compiling language at times rather than C. And being lazy, I can't be bothered trying to learn and remember a different syntax for the scripting language and the compiling language. So Lua for scripting and a Lua-like-syntax-wrapper-for-C is exactly what I'd like to have available at the moment (rather than Genie, with its Python-like wrapper for C). Well, Lua-syntax wrapper in preference to a Lua-like-syntax wrapper.DMcCunney wrote:Why?mcewanw wrote:Lua would be a better fit. But then we need a version of say Genie that also provides a Lua-like syntax.
Genie and Vala are essentially wrappers around C, providing a higher-level view of the system. But the output is C, to be compiled and built by GCC and friends, and you have to have the devx_XXX.sfs package installed to create your applications.
The issues I've seen bite are understanding parsing and precedence, and wrapping your mind around regular expressions.mcewanw wrote:Why... because nice though Python code looks I remember being bored stiff with the language (no idea why). But ... I dabbled much more recently with Lua and that interested me. And ... I'm lazy. I would also sometimes like to use a scripting language which had less of the idiosyncrasies endemic to bash; getting a bash script to work as planned can sometimes be a real pain to the brain.DMcCunney wrote:Why?mcewanw wrote:Lua would be a better fit. But then we need a version of say Genie that also provides a Lua-like syntax.
Genie and Vala are essentially wrappers around C, providing a higher-level view of the system. But the output is C, to be compiled and built by GCC and friends, and you have to have the devx_XXX.sfs package installed to create your applications.
I haven't seen one, and I don't expect to. Lua is aimed at a different class of problems. For instance, it's great for embedding in other programs to provide scripting capabilities. There's an editor called SciTE-it based on SciTE and the Scintill edit control that uses Lua as the extension language, and the author of SciTE-it finally abandoned that effort and created TextAdept, which is a small C core and a whole lot of Lua to provide functionality and extensibility. (I have both installed under Puppy.)But I would also like to use a higher level compiling language at times rather than C. And being lazy, I can't be bothered trying to learn and remember a different syntax for the scripting language and the compiling language. So Lua for scripting and a Lua-like-syntax-wrapper-for-C is exactly what I'd like to have available at the moment (rather than Genie, with its Python-like wrapper for C). Well, Lua-syntax wrapper in preference to a Lua-like-syntax wrapper.
No, I think you're right. I was misremembering, because AT&T at one point was rewriting the Unix kernel in C++.I'm pretty sure Linux is written in C and not C++
I used to teach Linux kernel programming and I don't think the basics has changed.
No, they don't. And we now have things like bdiff for binary files.mcewanw wrote:Very true in general, and in traditional UNIX "philosophy" - bash, Bourne, Korn and so on fitting in with that philosophy. Just didn't want anyone imagining that ASCII was an enforced limitation. There are many utilities which also work with binary strings and such utilities can easily be written in C for calling from bash. The command cmp for example compares files byte by byte; the files don't need to be ASCII files as far as I remember.DMcCunney wrote: It's not just bash (or the Bourne shell, C shell, Korn shell, Z shell, etc.). The various utilities you call from shell scripts also tend to assume ASCII strings. Awk, diff, grep, sed, tr and the like are intended to process ASCII files. As mentioned, the developers of Unix were software developers, and many of the utilities provided with a *nix system are intended to manipulate and transform source code in various ways.
I attended a talk once by Thomas Weinberger, the "W" in awk. He stated that awk was originally designed to do "one liners" at the command prompt to examine files and return a result, and he described his shock the first time he encountered a multi-page awk script.But the traditional shell, along with utilities such as those you mentioned, was designed and optimised for processing text files, which is a very transportable format (which, via the likes of the more recently thought up XML way of formatting text data can also be provided with a complex structure yet remain able to be processed/parsed/transformed with the old UNIX utils such as sed and awk). Awk's a particularly interesting utility too, of course, since it is a programming language in its own right.
