Thursday, May 1, 2008

Can There be a Viable Free Open Source Alternative to Magma, Maple, Mathematica, and Matlab?



For over a decade I have primarily done research in number theory that often involves computation, mainly using Magma. In 2004 I realized that it was stupid for me to continue building my work on Magma because Magma is proprietary, the development model is closed, Magma is expensive which is bad for students, and the language itself lacked many features (e.g., user defined classes) that I had requested repeatedly for over 5 years. Thus to continue to use only Magma unacceptably limited my potential in both research and education.

Having used Magma for many years, I simply could not switch to an existing open source system. The only serious free open source software for number theory is PARI, whose capabilities are far behind that of Magma in numerous critical areas of interest to me, including exact linear algebra, commutative algebra, and algebraic curves. And no other free system--GAP, Singular, Axiom, Maxima, Macaulay 2, etc.--even comes close in all these areas. In fact, after a decade of patiently waiting, I doubt they ever will.

Magma is the result of several decades of hard work by extremely talented fulltime mathematicians and programmers such as John Cannon, Allan Steel and Claus Fieker. I've worked with them and they are simply amazing. The situation seemed hopeless. If I had only never used Magma and tasted of the forbidden fruit...

In 2004, sure that there was no possible way to solve this problem, and driven by nothing but pure blind compulsion, I started the Sage project as my little free open source alternative to Magma, and spent an insane amount of time working on it even though I was utterly convinced that there was no hope of Sage ever succeeding. The first version of Sage consisted of the Python interpreter and a few scripts for doing number theory.

After a year, some of my first feedback from the computer algebra research community came from Richard Fateman on December 6, 2005 when he posted his opinion of the Sage project to sci.math.symbolic: ``By avoiding applications (say, to engineering design, finance, education, scientific visualization, etc etc) the activity is essentially doomed. Why? Government funding for people or projects will be a small percentage of the funding for pure mathematics. That's not much. And the future is pretty grim.''

Honestly, I believed he was right, but I just couldn't stop myself from working on Sage. It is now nearly three years later and the Sage project currently has around 100 active developers and 10,000 users. In November 2007, Sage won first place in the scientific category of the Trophees du Libre, which is a major international free software competition. Sage is funded by the US National Science Foundation, the US Department of Defense, the University of Washington, Microsoft Research, Google and private donations. Sage has new releases every two weeks, and typically 30--40 people contribute to each release. All new code contributions to Sage are peer reviewed, and every new function must be documented with tests that illustrate its usage. The docs have over 50,000 lines of input examples.

So what is Sage and what makes it unique? Sage is:

  1. a huge distribution of free open source mathematical software that is surprisingly easy to build from source,

  2. a set of interfaces to most other mathematical software systems, and

  3. a new Python library that fills in huge gaps in other open source math software included in Sage, unifies everything offering a smooth user experience, and provides a modern web-based graphical notebook interface with math typesetting and integrated 2D and 3D graphics.



Sage is the first large general purpose mathematics software system that uses a mainstream programing language (Python) as the end user language. Python--easily one of the world's top 10 programming languages--is a powerful and beautiful modern interpreted programming language with an organized and professional developer base and millions of users. Sage also makes extensive use of a Python-to-C compiler called Cython. Thus Sage has a tremendous advantage over every other general purpose computer algebra system, since Python has thousands of third party libraries, sophisticated support for object serialization, databases, distributed programming, and a major following in scientific computing.

Instead of reinventing the wheel, Sage combines many of the best existing open source systems that have been developed over the last 40 years (about million lines of code) with about 250,000 lines of new code. Every single copy of Sage includes all of the following software (and much much more):
  • Algebra and calculus: Maxima, SymPy
  • High precision arithmetic: GMP, MPFR, MPFI, quaddouble, Givaro
  • Commutative algebra: Singular
  • Number theory: PARI, NTL, mwrank, ECM, FLINTQS, GMP-ECM
  • Exact linear algebra: LinBox, IML
  • Group theory: GAP
  • Scientific computation: GSL, SciPy, NumPy, cvxopt
  • Statistical computation: R
  • Graphics (2d and 3d): Matplotlib, Tachyon3d, Jmol
Sage is thus the first system to combine together such a wide range of libraries and programs in a meaningful way. Instead this huge range of programs is tied together using Python's excellent extensibility via C libraries and using pseudo-tty's. Sage has a highly developed unified collection of pseudo-tty based interfaces that make it is possible to make extensive use of Maple, Mathematica, Magma, Matlab, GAP, Maxima, Singular, PARI, and many other systems from anywhere within a single Sage program.

Curious? If you want a viable open source alternative to Magma, Maple, Mathematica and Matlab, drop everything, try out Sage now and become a Sage developer.

http://www.sagemath.org

97 comments:

  1. I've used SAGE, but it lacks the only feature I care about on Matlab: Simulink.

    Matlab is great, because it's focused on the real industry: Control Systems, bind with MS Excel and Databases, LabView bind, compiler, data acquisition modules, video acquisition,and so on.

    It's true Sage it's a great software, but if it doesn't implement what the average engineer uses on their work, it's going to be a classroom toy or research utility forever, and will never grow as Matlab.

    Take as example R and SciLab, both cares about what the Engineers need, and both of them are very successfull in the industry.

    ReplyDelete
  2. Hey! Sebastian - there's this new cool thing called 'open source' - it allows you fix whatever bugs you by and for your self.

    ReplyDelete
  3. do you really think I have time to write my own implementation of simulink on Sage?, and of course, once I'm done, I'll rely on it for simulating that controller for that inexpensive $2M reactor I'm working on.

    By the way, do you know what is Simulink?

    ReplyDelete
  4. I think SAGE is one of the best free software I've ever seen. I'm an italian guy and I study math, I discovered this software about 6 months ago, writing my thesis over ECC, ECPP & related algorithms, so you can easily understand why I think that SAGE is great. I don't know if, today, it can be a complete alternative to Mathematica, Matlab, etc., but I costantly see new releases, with amazing improvements (always with a complete documentation). I love using my browser as GUI, the use of Python as end-user language, and all the things you mentioned...what else?
    I really believe in this project.

    ReplyDelete
  5. I've used Matlab for a long time now. I can see where Sage would be of great use, especially with making compiled code easily with Cython.

    Matlab is a bit porky with small, tight inner loops that can't be vectorised, so this can potentially of great help!

    And yes, simulink analogue would be nice too. Of course you can just fork out the money and buy it, but writing it yourself and you might also learn something (and profit!).

    ReplyDelete
  6. I'm glad to hear that you've found a project that you feel comfortable further developing. No doubt your input will help bring Sage closer to your ideals and thus distribute any educational and / or research benefits to the global community (I imagine that many - like you used to - are waiting for them). While I realize that it might not mean much, please accept my gratitude and respect for doing work we will all benefit from.

    ReplyDelete
  7. FYI, good showmedo tutorial here:

    here

    ReplyDelete
  8. Hey Sebastian,

    You do make excellent points both about the lack of features you require in SAGE and the unrealistic expectation that you should fix them yourself just because the system is open source.

    However you do point out one interesting problem when you ask the other poster if they know what Simulink is. I don't know if they do, and I'm not a Sage developer (or a Matlab user), but I don't know what Simulink is.

    You might help bring about the implementation of the features you desire by mentioning them (which you have done), and explaining what they are and why they're important (which you haven't done so much). Could you explain Simulink in a nutshell? Is it the MS Excel and database bindings you mentioned? Does SAGE have *any* of the capabilities you are looking for here which could be expanded, or is Simulink something completely different from whats in SAGE?

    You might also want to share these thoughts directly with the developers either through their email or through a mailing list, as well as in these blog comments. After all, Stein in his background on SAGE mentions that a big part of his motivation for developing it was the lack of responsiveness of the developers of Magma. He'd probably be sympathetic with your requests for improvements, and you might see them happen.

    ReplyDelete
  9. Full disclosure: I am a Sage developer.

    Of the four commercial products mentioned that Sage wants to become a viable alternative to the features provided by Matlab, especially the tools engineers care about, are certainly the most immature bits in Sage. Numpy and Scipy can do many things that Matlab users want, but the integration with Sage needs to and will become better and as more people with an engineering background start using Sage or become Sage developers this will improve. Sage is certainly strongest at the moment in more abstract algebraical things like number theory, but that is mostly since Sage was started by a number theorist.

    Cheers,

    Michael

    ReplyDelete
  10. It's true that the state of the art in open source is often well behind commercial offerings because it is often the product of volunteer labor.

    However, I doubt if a $2 million reactor design is flawed due to a bug in Matlab, Mathworks will not be compensating you for the problem. At best, you might get your money back, so your software will be gratis and flawed anyways. It is also unlikely that a company will fear having their reputation soured by such an event unless that company makes some extremely egregious error.

    It is exciting to see the critical mass of scientific software that is taking shape, especially around Python. I use plenty of proprietary software such as Matlab, Labview, and Mathematica and I am continually frustrated at the clumsy ways, at best, with which one has to get these programs to interoperate. By basing programs on Python classes, hopefully one can just pick and choose the parts needed and stitch them into a single program.

    ReplyDelete
  11. The current reality is that there are two specialization areas: symbolic math (Mathematica, Maxima, Sage, etc) and numerical (mostly linear-algebra-based systems such as Matlab, Octave, Scilab. R, IDL, NumPy/SciPy, etc). Sebastian really uses Matlab's control theory toolbox Simulink, which is famous for having a GUI where you drag-and-drop standard control loop elements and link them in a graphical window. Simulink is well known and well executed; Scilab has a FOSS clone of it.

    I would suggest looking at Octave, which is a FOSS Matlab clone. It implements many control theory functions of Matlab, but it doesn't have it's GUI. Still, someone working in this area on Matlab should be able to use Octave productively as well.

    Octave is quite well established (it is a standard Debian and RPM package available easily in their respective distributions. It is also available for Windows and Mac
    from the Octave web site

    ReplyDelete
  12. daniel said:
    "...I am continually frustrated at the clumsy ways, at best, with which one has to get these programs to interoperate. By basing programs on Python classes, hopefully one can just pick and choose the parts needed and stitch them into a single program."

    Not just "hopefully" -- it actually works, and this is why i like Sage!

    One of my projects is about cohomology rings of finite p-groups. It comprises:
    - groups: Sage contains GAP!
    - C-programs written by my boss: Sage can incorporate them due to Cython!
    - Gröbner bases in graded-commutative rings: Sage contains Singular!
    - my own code: Sage has a decent programming language, namely Python!

    And all these components work together in a very smooth way.
    "Smooth" means: If you define an ideal I in a multivariate polynomial ring, then I.groebner_basis() computes a groebner basis. It uses Singular in the background, via an interface, but on the surface it "feels" like ordinary python.

    And, if you have a Magma licence, you also have an interface that makes Magma usable within Sage.

    Finally, Sage is open source. This is how i came into mathematical computations: Look at the code that intelligent people produced, and learn...

    ReplyDelete
  13. There's a pedagogical element that hasn't been discussed much in this thread. I'm an undergraduate math student just on the verge of grappling with any substantial limb of the mathematical field. I'm also interested on computation, practical and theoretical. As I'm reviewing my options for capstone and personal projects, I am bearing in mind how to ensure that such project test my mathematical aptitude, provide grounds for expanding my understanding and interest, as well as create a basis on which future (educational and other) pursuits might be based.

    I've been a fan, but not quite a user, of SAGE for at least a year now. I keep up, more or less, with the groups and the releases. It recently occurred to me that I could hit all these points by building a project around SAGE, with the very nice bonus that I could contribute to a project I greatly admire. Further, developing a facility with Python in conjunction with its fruitful application to mathematics makes my CV a bit more impressive. This last isn't trivial; I'm a nontraditional student with a family, a mortgage, and all the other trappings of adult life. One of the shorter paths to interesting and fulfilling endeavors and employment, for me, includes SAGE.

    That is to say that, in addition to its growing utility for established mathematicians and engineers, SAGE offers a singular opportunity for students to learn and contribute to the mathematical community. I don't anticipate Wolfram, Mathworks, etc. being so open to user-contributed work; and the licensing wouldn't be open.

    Just my two cents. Thanks to the SAGE team.

    Daniel Black

    ReplyDelete
  14. I don't have doubts about Sebastian's real need to use Simulink or other tools. But often I have observed that managers in companies or even some people in academic institutions buy packages like the M-tools simply because someone else is paying for it. There is also the belief that simply because you are paying a lot of money, the package must be more reliable (than an academic toy developed by some hobbyists for classroom demo). But most such thinking is quite shortsighted or completely wrong. A lot of companies and academic institutions can save a lot of money (while getting some rewarding work done) by actively incorporating and developing tools like Sage in their institutions.

    There are other issues which engineers would probably not appreciate. They are happy designing their reactors or aircraft wings using the M-tools. But in basic sciences such as pure maths, we simly cannot accept a computer assisted proof if the programs were developed in one of the M-tools especially if we cannot verify these results by other means. For this idealogical reason alone it is worth the effort to develop tools such as Sage. And for all you know, you might not be able to run your own programs after a few years if you wrote your programs in one of the proprietary tools. Are your results valid only as long as you have a license to the tool you used? I wonder how many people have such irreproducible results in their collected works. I think results obtained with open source tools will have much longer shelf life - beyond the life of the open source tools themselves.

    Personally Sage inspires me. Just following their sage-devel group is quite stimulating. In my work I never had to compute, but Sage makes me want to compute. Now when I prove a result or before I prove it, I think "can I compute with a few small examples and illustrate my result or conjecture?". I think I will attempt to do that and I hope I will contribute to such a project some day.

    I am greatly optimistic that Sage will be successful like TeX, linux, Wikipedia, ...

    ReplyDelete
  15. hi, Ive been following SAGE for a year now, and I have been tinkering around in it.

    BUT, as an electronics engineer currently working in photonics, there is really not much in sage that could make my life easier, or replace the dominant role that MATLAB has played in most of my computational work.

    However, since I started on my masters degree in photonics, I observe that a lot of the tools developed my the research group I work in are in python. they basically involve solving maxwell's equations in various structures and plotting the results and are based on numpy, scipy, matplotlib etc. Given this, I was surprised by how difficult it was to find a good way of numerically solving differential equation in sage. there is never more than one short page about it in the documentation, and its never terribly useful.

    really, some one needs to look into this, and do something for us engineers. Id try and contribute myself, but to be honest, Im not much of a coder. would be nice to try though.

    and keep up the good work !

    ReplyDelete
  16. I'm not sure what DEs Psmith (presumably a Wodehouse fan:-) wants
    but there is a preliminary draft of a book on DEs and Sage at
    http://sage.math.washington.edu/home/wdj/teaching/index.html
    That is certainly more than "one short page" but may not be useful.
    If he wants numerical solns of PDEs then I can't help him.
    In fact, the only higher order PDEs dealt with are the heat and wave equations.
    DEs is a big topic and it would be helpful if he would explain what
    he wanted.

    ReplyDelete
  17. This comment has been removed by a blog administrator.

    ReplyDelete
  18. This comment has been removed by a blog administrator.

    ReplyDelete
  19. Self praise is no recommendation.

    Maybe the viable free open source alternative to M,M,M,M is Maxima, for symbolic math. Octave for Matlab, and GAP for Magma. Actually, I have no particular use for Magma or GAP, and so the differences between them are not as important to me as to Prof. Stein.

    One important difference to some people: Maxima runs natively on Windows; Sage does not.

    As for the future, time will tell.

    Richard Fateman

    ReplyDelete
  20. This comment has been removed by a blog administrator.

    ReplyDelete
  21. This comment has been removed by a blog administrator.

    ReplyDelete
  22. This comment has been removed by a blog administrator.

    ReplyDelete
  23. This comment has been removed by a blog administrator.

    ReplyDelete
  24. This comment has been removed by a blog administrator.

    ReplyDelete
  25. This comment has been removed by a blog administrator.

    ReplyDelete
  26. This comment has been removed by a blog administrator.

    ReplyDelete
  27. This comment has been removed by a blog administrator.

    ReplyDelete
  28. This comment has been removed by a blog administrator.

    ReplyDelete
  29. This comment has been removed by a blog administrator.

    ReplyDelete
  30. This comment has been removed by a blog administrator.

    ReplyDelete
  31. This comment has been removed by a blog administrator.

    ReplyDelete
  32. This comment has been removed by a blog administrator.

    ReplyDelete
  33. This comment has been removed by a blog administrator.

    ReplyDelete
  34. This comment has been removed by a blog administrator.

    ReplyDelete
  35. This comment has been removed by a blog administrator.

    ReplyDelete
  36. This comment has been removed by a blog administrator.

    ReplyDelete
  37. This comment has been removed by a blog administrator.

    ReplyDelete
  38. This comment has been removed by a blog administrator.

    ReplyDelete
  39. This comment has been removed by a blog administrator.

    ReplyDelete
  40. This comment has been removed by a blog administrator.

    ReplyDelete
  41. This comment has been removed by a blog administrator.

    ReplyDelete
  42. This comment has been removed by a blog administrator.

    ReplyDelete
  43. This comment has been removed by a blog administrator.

    ReplyDelete
  44. This comment has been removed by a blog administrator.

    ReplyDelete
  45. This comment has been removed by a blog administrator.

    ReplyDelete
  46. This comment has been removed by a blog administrator.

    ReplyDelete
  47. This comment has been removed by a blog administrator.

    ReplyDelete
  48. This comment has been removed by a blog administrator.

    ReplyDelete
  49. This comment has been removed by a blog administrator.

    ReplyDelete
  50. This comment has been removed by a blog administrator.

    ReplyDelete
  51. This comment has been removed by a blog administrator.

    ReplyDelete
  52. This comment has been removed by a blog administrator.

    ReplyDelete
  53. This comment has been removed by a blog administrator.

    ReplyDelete
  54. This comment has been removed by a blog administrator.

    ReplyDelete
  55. This comment has been removed by a blog administrator.

    ReplyDelete
  56. This comment has been removed by a blog administrator.

    ReplyDelete
  57. This comment has been removed by a blog administrator.

    ReplyDelete
  58. This comment has been removed by a blog administrator.

    ReplyDelete
  59. This comment has been removed by a blog administrator.

    ReplyDelete
  60. This comment has been removed by a blog administrator.

    ReplyDelete
  61. This comment has been removed by a blog administrator.

    ReplyDelete
  62. This comment has been removed by a blog administrator.

    ReplyDelete
  63. This comment has been removed by a blog administrator.

    ReplyDelete
  64. This comment has been removed by a blog administrator.

    ReplyDelete
  65. This comment has been removed by a blog administrator.

    ReplyDelete
  66. This comment has been removed by a blog administrator.

    ReplyDelete
  67. coats & jackets
    If one knows where to, the world will give way.
    north face jackets
    moncler
    World-class brands
    moncler jackets
    The top fabric produced in France is 100%
    duvetica
    Italy's top outdoor brands
    peuterey
    Good quality and function, more accord with fashionable design
    spyder jackets
    High-grade, innovation, the trend of the brand
    Columbia Sportswear
    The quality is so good
    quiksilver jackets
    Young and creative style
    ski jacket


    You can have a look at it.
    jordan shoes
    Wow.
    newest jordans
    jordan 7
    Wonderful!
    air yeezy
    jordan true flight
    If you like,you can contact us.
    jordan 3
    jordan 4
    We offer different styles.
    jordan 1
    jordan 2
    Thanks.
    jordan 5
    jordan ajf shoes
    There are cheap shoes to choose
    nike footwear
    jordan flight 45
    Good quality with low price.
    air jordan 2010
    Air Jordan 2009
    Enjoy it!
    jumpman
    nike trainers

    ReplyDelete
  68. This comment has been removed by a blog administrator.

    ReplyDelete
  69. This comment has been removed by a blog administrator.

    ReplyDelete
  70. This comment has been removed by a blog administrator.

    ReplyDelete
  71. This comment has been removed by a blog administrator.

    ReplyDelete
  72. This comment has been removed by a blog administrator.

    ReplyDelete
  73. This comment has been removed by a blog administrator.

    ReplyDelete
  74. This comment has been removed by a blog administrator.

    ReplyDelete
  75. This comment has been removed by a blog administrator.

    ReplyDelete
  76. This comment has been removed by a blog administrator.

    ReplyDelete
  77. This comment has been removed by a blog administrator.

    ReplyDelete
  78. This comment has been removed by a blog administrator.

    ReplyDelete
  79. This comment has been removed by a blog administrator.

    ReplyDelete
  80. This comment has been removed by a blog administrator.

    ReplyDelete
  81. This comment has been removed by a blog administrator.

    ReplyDelete
  82. This comment has been removed by a blog administrator.

    ReplyDelete
  83. This comment has been removed by a blog administrator.

    ReplyDelete
  84. This comment has been removed by a blog administrator.

    ReplyDelete
  85. This comment has been removed by a blog administrator.

    ReplyDelete
  86. This comment has been removed by a blog administrator.

    ReplyDelete
  87. This comment has been removed by a blog administrator.

    ReplyDelete
  88. This comment has been removed by a blog administrator.

    ReplyDelete
  89. This comment has been removed by a blog administrator.

    ReplyDelete
  90. This comment has been removed by a blog administrator.

    ReplyDelete
  91. This comment has been removed by a blog administrator.

    ReplyDelete
  92. This comment has been removed by a blog administrator.

    ReplyDelete
  93. This comment has been removed by a blog administrator.

    ReplyDelete
  94. This comment has been removed by a blog administrator.

    ReplyDelete
  95. This comment has been removed by a blog administrator.

    ReplyDelete
  96. This comment has been removed by a blog administrator.

    ReplyDelete