Sage Days 15: What is on the Horizon in Sage Development
system:sage

<h1 style="text-align: center;"><br /></h1>
<h1 style="text-align: center;"><br /></h1>
<h1 style="text-align: center;"><br /></h1>
<h1 style="text-align: center;"><br /></h1>
<h1 style="text-align: center;">What is on the Horizon for Sage 5.0<br /></h1>
<h2 style="text-align: center;"><span style="color: #333333;">William Stein</span></h2>
<h2 style="text-align: center;"><span style="color: #333333;">May 2009 at Sage Days 15</span></h2>
<p style="text-align: center;"><span style="color: #333333;"><img src="horizon.jpg" alt="" width="516" height="387" /><br /></span></p>

{{{id=7|

///
}}}

<h2>Outline of Talk</h2>
<ol>
<li><strong>Native Ports</strong>: to Solaris, Windows, OS X 64-bit; also SPD</li>
<li><strong>The Sage Notebook</strong>: rewrite core to better support embedding in web pages, test framework, better architecture, and LaTeX integration</li>
<li><strong>Symbolic Calculus</strong>: switch to Pynac; implement symbolic algorithms in Pynac/Sage</li>
<li><strong>2d and 3d Graphics: </strong>add more features and support</li>
<li><strong>Statistics</strong>: create a natural, clean, native interface to stats functionality</li>
<li><strong>Combinatorics and Graph theory</strong>: finish porting of MuPAD-combinat to Sage; remove Symmetrica from Sage; beyond Networkx; implement group theory code natively in Sage instead of GAP</li>
<li><strong>Arithmetic</strong>: many optimizations across the spectrum</li>
<li><strong>Number Theory: </strong>quaternion algebras, quadratic forms, modular forms, descent on elliptic curves, Heegner points, $L$-functions</li>
<li><strong>Algebraic Geometry: </strong>fully doctest all the schemes directories; add new functionality (especially for <em>curves</em>)</li>
</ol>

{{{id=6|

///
}}}

<h2 style="text-align: center;">Native Ports</h2>
<h3 style="text-align: center;"><span style="color: #333333;">to Solaris, Windows, OS X 64-bit; also SPD</span></h3>
<p><span style="color: #333333;"><br /></span></p>

{{{id=27|

///
}}}

<p style="text-align: center;"><img src="solaris.png" alt="" width="295" height="154" /></p>
<p style="text-align: center;">&nbsp;</p>
<p><strong>Solaris</strong>: Various people have worked off and on very hard to port Sage to Solaris since <strong>late 2005!&nbsp; </strong>Finally, after years of work, and thanks to over a year of hard work almost entirely by Michael Abshoff (100% funded by the US Dept of Defense), the 32-bit port of Sage to Solaris 10 (both Sparc and x86) is finally done!&nbsp; This involved everything from getting rid of CLISP (replaced by ECL) to fixing all kinds of weird bugs all over the place.&nbsp;&nbsp; Sun Microsystems has and is expressing strong interest in this work, and just donating a high-end sparc machine to the project.</p>
<p><strong>Solaris 64-bit: </strong>&nbsp; The port of Sage to 64-bit Solaris remains, but should be fairly straightforward given the work Michael Abshoff did on OS X 64-bit (another bi-arch platform).</p>
<p>&nbsp;</p>
<p>&nbsp;</p>

{{{id=30|

///
}}}

<p>&nbsp;</p>
<p style="text-align: center;"><img src="leopard-logo.jpg" alt="" width="170" height="170" /></p>
<p>&nbsp;</p>
<p><strong>OS X 64-bit and OS X 10.6:&nbsp; </strong>The port of Sage to 64-bit OS X is now done due almost entirely to Michael Abshoff (with important help from Martin Albrecht this week), but there are still likely to be numerous kinks to work out as it gets broad testing.&nbsp; Also, the next version (10.6) of OS X is coming soon.&nbsp; Everybody using OS X intel, should switch to fully using the 64-bit version as soon as possible.</p>
<p>Advantages of 64-bit: you can use far more RAM; many basic arithmetic operations are <em><strong>over twice as </strong></em>fast, which impacts high level algorithms:</p>
<p><strong>&nbsp;&nbsp;&nbsp;&nbsp; 32-bit OS X:</strong></p>
<p>sage: time n = factorial(10^6)<br />CPU times: user 2.02 s, sys: 0.14 s, total: 2.16 s</p>
<p>age: time f = ZZ['x'].random_element(degree=100000)^2<br />CPU times: user 0.38 s, sys: 0.10 s, total: 0.48 s</p>
<p>sage: time set_random_seed(0); a=random_matrix(ZZ,300).det()<br />CPU times: user 0.74 s, sys: 0.08 s, total: 0.82 s</p>
<p><strong>&nbsp;&nbsp;&nbsp;&nbsp; 64-bit OS X (same computer):</strong></p>
<p>sage: time n = factorial(10^6)<br />CPU times: user 0.79 s, sys: 0.10 s, total: 0.89 s</p>
<p>sage: time f = ZZ['x'].random_element(degree=100000)^2<br />CPU times: user 0.16 s, sys: 0.04 s, total: 0.19 s</p>
<p>sage: time set_random_seed(0); a=random_matrix(ZZ,300).det()<br />CPU times: user 0.62 s, sys: 0.04 s, total: 0.66 s</p>

{{{id=34|

///
}}}

<p>&nbsp;</p>
<p style="text-align: center;"><img src="windows-vista-logo.jpg" alt="" width="232" height="279" /></p>
<p style="text-align: left;"><strong>Microsoft Windows</strong>:</p>
<ul>
<li>Windows users currently use VMware or http://sagenb.org with success.&nbsp; Sage supports Windows as a web client pretty well.&nbsp; For example, twice a week I teach my Sage course in a lab full of <em><strong>generic windows machines</strong></em> and all the students use a single Sage notebook server running off a single dual-processor VMware image.&nbsp; I'm suspect the students in fact <strong><em>prefer</em></strong> this to having Sage installed locally on all the lab machines, since they have easy access to all their work anywhere they go.&nbsp;</li>
<li>Overall Sage download stats since Feb 15 (thanks to Harald Schilly):</li>
</ul>
<p><span style="font-family: terminal,monaco;">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 1. &nbsp; &nbsp; &nbsp;microsoft_windows&nbsp; 5,661&nbsp;&nbsp;&nbsp;&nbsp; &lt;--- vmware, by *far* the most popular !!<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 2. &nbsp; &nbsp; &nbsp;linux/32bit &nbsp; &nbsp;&nbsp;&nbsp;&nbsp; 3,077<br />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 3. &nbsp; &nbsp; &nbsp;src &nbsp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 1,964<br /> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 4. &nbsp; &nbsp; &nbsp;apple_osx/intel&nbsp;&nbsp;&nbsp; 1,404<br /> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 5. &nbsp; &nbsp; &nbsp;linux/64bit &nbsp; &nbsp;&nbsp;&nbsp;&nbsp; 1,070<br /> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 6. &nbsp; &nbsp; &nbsp;apple_osx/powerpc &nbsp;&nbsp; 372<br /> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 7. &nbsp; &nbsp; &nbsp;linux/atom &nbsp; &nbsp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 122<br /> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 8. &nbsp; &nbsp; &nbsp;apple_osx/ppc &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 98<br /> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 9. &nbsp; &nbsp; &nbsp;solaris&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 15<br /> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 10. &nbsp; &nbsp;&nbsp; linux/itanium &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 14</span></p>
<p><span style="font-family: terminal,monaco;">Total.............................. 13797 (or just under 5000/month)<br /></span></p>
<p>&nbsp;</p>
<p><span style="font-family: terminal,monaco;"><span style="font-family: times new roman,times;">I started a</span></span><span style="font-family: terminal,monaco;"><span style="font-family: times new roman,times;"><strong> "fully native"</strong> port of Sage to Windows in mid-February 2009.&nbsp; It is a new project and growing organically <strong><em>just like</em></strong> Sage-for-UNIX did.&nbsp;&nbsp; See <a href="http://windows.sagemath.org" target="_blank">http://windows.sagemath.org</a> and my talk on Tuesday.</span></span></p>
<ul>
<li><span style="font-family: terminal,monaco;"><span style="font-family: times new roman,times;"><strong>Currently includes</strong>: </span></span>bzip2, cython, docutils, freetype, ipython, libpng, matplotlib, mercurial, m4ri, moin, mpir, networkx, ntl (as WinNTL), numpy, pycrypto, pyreadline, python, pywin32, scons, setuptools, sphinx (with dependencies: jinja2, pygments), sqlalchemy, sqlite, sympy, twisted, wexpect, zlib, zodb (with dependencies: zope.interface, zope.proxy, zope.testing, zconfig, and zdaemon).&nbsp; </li>
<li><strong>MS Visual Studio 2008</strong>: Everything builds from scratch on Windows XP or Windows Vista using MSVC 2008.</li>
<li><strong>Contributors so far</strong>: almost entirely Dan Shumow and I, but there is more brewing.&nbsp; E.g., Blair Sutton wrote a new Python-based build system and contributed it.&nbsp; Some help from Tom Boothby, Chris Gorecki, William Cauchois.</li>
<li>Microsoft Research funding helped.</li>
<li><strong>Next steps:</strong> (1) get scipy in, and (2) get the notebook with preparser <strong><em>ported</em></strong>, which will provide a viable alternative to all the other existing Windows scientific Python distros (Enthought Tool Suite, Python(x,y), etc.).&nbsp; </li>
<li><strong>Cygwin/SFU: </strong>In parallel with the above, get Sage to build and work in Cygwin and/or using Services for Unix (Microsoft's analogue of Cygwin). Michael Abshoff has put the most effort into this direction.</li>
<li><strong>REvolution</strong>: they did a contract with Microsoft, and fairly quickly produces a full native (64-bit) version of R for Windows.&nbsp; </li>
</ul>

{{{id=59|

///
}}}

{{{id=33|

///
}}}

<p><strong>SPD = Source Python Distribution</strong></p>
<p style="text-align: center;"><strong><img src="ondrej_certik.jpg" alt="" width="147" height="156" /><br /></strong></p>
<ul>
<li>Project that <em><strong>Ondrej Certik</strong></em> started that is basically a "lightweight Sage distribution for numerical computation". </li>
<li><a href="http://code.google.com/p/spdproject/" target="_blank">http://code.google.com/p/spdproject/</a></li>
<li>Superb way to make the work we've done with Sage available to a wider range of users. </li>
<li>Will provide "for free" numerous fixes back to Sage.</li>
<li>I'm very enthuisiastic about this project.</li>
</ul>

{{{id=42|

///
}}}

{{{id=41|

///
}}}

{{{id=32|

///
}}}

{{{id=4|

///
}}}

<h2 style="text-align: center;"><span style="color: #000000;">The Sage Notebook</span></h2>
<h3 style="text-align: center;"><span style="color: #333333;">rewrite core to better support embedding in web pages, test framework, better architecture, and LaTeX integration</span></h3>

{{{id=38|

///
}}}

<ul>
<li><strong>Sage lite:</strong> Mike Hansen and I recently split the notebook off from Sage, but we need to get this patch into Sage (ASAP): See trac <a href="http://trac.sagemath.org/sage_trac/ticket/5789" target="_blank">#5789.</a>&nbsp; It is currently bit rotting. </li>
<li><strong>Another major rewrite:</strong> Mike Hansen has been experimenting with rewriting some of the core of the notebook to use a general web framework. </li>
<li><strong>Testing:</strong> We need a better automatic testing system.&nbsp; Lack of testing is perhaps the biggest obstruction to notebook development. (?)</li>
<li><strong>LaTeX</strong>: Users love the LaTeX integration in the Sage notebook.&nbsp; It just needs more polish (John Palmierri)</li>
<li><strong>Notebook servers</strong> are popping up all over the place.&nbsp;&nbsp;&nbsp; Let's make them easier to configure and deploy.&nbsp; (Jason Grout)</li>
<li><strong>Embeded interacts</strong>: ability to easily and robustly embed interacts in any random web page.&nbsp; I need this for the modular forms and $L$-functions database project. (Mike Hansen will do this soon)</li>
</ul>

{{{id=37|

///
}}}

{{{id=28|

///
}}}

{{{id=10|

///
}}}

<h2 style="text-align: center;">Symbolic Calculus</h2>
<h3 style="text-align: center;"><span style="color: #333333;"> switch to Pynac; implement symbolic algorithms in Pynac/Sage</span></h3>
<p>In any technical field, not <em><strong>understanding and controlling</strong></em> ones core tools leads to second-rate results.&nbsp; That has been the situation with symbolics across the board in Sage, due to wrapping Maxima.&nbsp;</p>
<ol>
<li>The <strong>current pure Python/Maxima hybrid implementation</strong> (due mostly to Bobby Moretti, as an undergrad student project) was always meant only as a reference implementation.</li>
<li><strong>Gary Furnish</strong> spent nearly a year by himself trying to rewrite everything in Cython from scratch, but this didn't work out.&nbsp; Nonetheless, we learned a lot from this effort.</li>
<li><strong>Burcin</strong> suggested somehow using <a href="http://www.ginac.de/" target="_blank">Ginac</a>, and with a few weeks very hard work, I found a way to completely strip Ginac's <a href="http://www.ginac.de/CLN/" target="_blank">CLN</a> (all basic arithmetic types) dependency and replace it by Python objects. </li>
<li><strong>Ginac</strong> is a mature, healthy, lively project, with regular releases, a native Windows version, etc.&nbsp; The code is easy to read and adapt.&nbsp; It is a solid foundation on which to build.</li>
<li><strong>Burcin, me, Mike Hansen, Robert Bradshaw, and Nick Alexander</strong> -- all working to switch Sage to use Pynac.&nbsp; This is a herculean task.</li>
</ol>
<p>&nbsp;</p>
<p>&nbsp;</p>
<p style="text-align: center;"><img src="ginac.png" alt="" width="225" height="109" /></p>
<p style="text-align: center;">(but it is a fine library on which to build a CAS)</p>

{{{id=43|

///
}}}

<h3>Where To?</h3>
<ul>
<li><strong>Low level symbolic arithmetic</strong>: All basic symbolic manipulation will switch to using the <em><strong>Pynac C++ library</strong></em> (=<a href="http://www.ginac.de/" target="_blank">Ginac</a> with Python data types) as soon as possible.&nbsp; This is potentially <em><strong>dramatically </strong></em>(100s of times) faster than what we get from Maxima, and is far easier for us to improve and extend.</li>
<li><strong>Eliminate Maxima depedency:</strong> All other symbolic calculus functionality will be implemented via Cython/C++/Sympy/Sage with the eventual goal to <em>completely eliminate the dependency on Maxima</em>: In each case, the goal will be to rethink the implementations using modern ideas, tools, libraries, etc.&nbsp; This will lead to interesting new results, speedups, etc.:&nbsp;      
<ul>
<li> Equality testing (using interval arithmetic -- in progress work of Robert Bradshaw)</li>
<li>Solving equations for a variable</li>
<li>Taylor expansion (sympy does a lot already)</li>
<li>Symbolic integration (implement numerous heuristics, plus a heuristic Riesch)</li>
<li>Factorization (via Singular, etc.)</li>
<li>Formal infinite sums (Burcin's Ph.d. thesis)</li>
</ul>
</li>
</ul>

{{{id=45|

///
}}}

{{{id=3|

///
}}}

<h2 style="text-align: center;">2d and 3d Graphics</h2>
<h3 style="text-align: center;"><strong> </strong><span style="color: #333333;">add more features and support</span></h3>

{{{id=47|

///
}}}

<div id="cell_text_48" class="text_cell">
<p><strong>3d Plotting</strong></p>
<ul>
<li><strong>implicit_plot3d: </strong>Carl Witty and Bill Cauchois recently implemented this, thus filling in a glaring gap in Sage's 3d plotting functionality (compared to Maple and Mathematica).</li>
<li><strong>doctests: </strong>Bill Cauchois greatly improved doctest coverage, though much still remains to do here</li>
<li><strong>Mathematica's</strong><strong> </strong>3d plotting<strong> </strong>still has numerous options than Sage's doesn't -- this needs to be fixed via hard work</li>
<li><strong>Jmol --</strong> isn't as rock solid as we wish it were -- I'm not sure how to improve matters.</li>
<li><strong>Resize</strong> -- given jQuery, being able to drag and resize a 3d plot would be easy to implement, but nobody has got around to it.</li>
<li><strong>Tick marks </strong>-- currently no control over axes labels or tick marks in 3d plots</li>
<li><strong>Interaction</strong> -- make it easy for people to <em><strong>manipulate</strong></em> elements of 3d plots, select points, etc., and have this interact with Sage.</li>
</ul>
</div>

{{{id=46|
var('x,y,z')
implicit_plot3d(cos(x)*sin(y) + cos(y)*sin(z) + cos(z)*sin(x), 
                (x,-2*pi,2*pi),(y,-2*pi,2*pi),(z,-2*pi,2*pi), color='red')
///
}}}

{{{id=54|
var('x,y,z')
F = (x^2+y^2+z^2+2*y-1)*((x^2+y^2+z^2-2*y-1)^2-8*z^2)+16*x*z*(x^2+y^2+z^2-2*y-1)
implicit_plot3d(F, (x,-3,3),(y,-3,3),(z,-3,4), opacity=.7, )
///
}}}

{{{id=50|

///
}}}

<p><strong>2d Plotting</strong></p>
<ul>
<li><strong>refactoring</strong> -- Mike Hansen has (and is still) refactoring the code to make it much cleaner and easier to work with</li>
<li><strong>doctesting </strong>--<span class="gI"> Karl-Dieter Crisman recently greatly improved the doctest coverage of 2d plotting</span></li>
<li><span class="gI"><strong>tick marks </strong>-- currently poor control over axes labels or tick marks in 2d plots<br /></span></li>
<li><span class="gI"><strong>interactivity in the browser (!) </strong>-- massive recent interest in FLOT, Canvas, Network Plotting in Javascript, etc., to make the 2d graphics in the Sage notebook far more interactive and flexible.&nbsp; This is the main next thrust for plotting.<br /></span></li>
</ul>

{{{id=53|
@interact
def _(n=(10..500)):
    r = random
    show(sum(circle((r(),r()),r()/10,fill=True,rgbcolor=(r(),r(),r()),alpha=0.5) for _ in range(n)),
         aspect_ratio=1,frame=True,gridlines=True)
///

<html><!--notruncate--><div padding=6 id='div-interact-53'> <table width=800px height=20px bgcolor='#c5c5c5'
                 cellpadding=15><tr><td bgcolor='#f9f9f9' valign=top align=left><table><tr><td align=right><font color="black">n&nbsp;</font></td><td><table><tr><td>
        <div id='slider-n-53' class='ui-slider ui-slider-3' style='margin:0px;'><span class='ui-slider-handle'></span></div>
        </td><td><font color='black' id='slider-n-53-lbl'></font></td></tr></table><script>(function(){ var values = ["10","11","12","13","14","15","16","17","18","19","20","21","22","23","24","25","26","27","28","29","30","31","32","33","34","35","36","37","38","39","40","41","42","43","44","45","46","47","48","49","50","51","52","53","54","55","56","57","58","59","60","61","62","63","64","65","66","67","68","69","70","71","72","73","74","75","76","77","78","79","80","81","82","83","84","85","86","87","88","89","90","91","92","93","94","95","96","97","98","99","100","101","102","103","104","105","106","107","108","109","110","111","112","113","114","115","116","117","118","119","120","121","122","123","124","125","126","127","128","129","130","131","132","133","134","135","136","137","138","139","140","141","142","143","144","145","146","147","148","149","150","151","152","153","154","155","156","157","158","159","160","161","162","163","164","165","166","167","168","169","170","171","172","173","174","175","176","177","178","179","180","181","182","183","184","185","186","187","188","189","190","191","192","193","194","195","196","197","198","199","200","201","202","203","204","205","206","207","208","209","210","211","212","213","214","215","216","217","218","219","220","221","222","223","224","225","226","227","228","229","230","231","232","233","234","235","236","237","238","239","240","241","242","243","244","245","246","247","248","249","250","251","252","253","254","255","256","257","258","259","260","261","262","263","264","265","266","267","268","269","270","271","272","273","274","275","276","277","278","279","280","281","282","283","284","285","286","287","288","289","290","291","292","293","294","295","296","297","298","299","300","301","302","303","304","305","306","307","308","309","310","311","312","313","314","315","316","317","318","319","320","321","322","323","324","325","326","327","328","329","330","331","332","333","334","335","336","337","338","339","340","341","342","343","344","345","346","347","348","349","350","351","352","353","354","355","356","357","358","359","360","361","362","363","364","365","366","367","368","369","370","371","372","373","374","375","376","377","378","379","380","381","382","383","384","385","386","387","388","389","390","391","392","393","394","395","396","397","398","399","400","401","402","403","404","405","406","407","408","409","410","411","412","413","414","415","416","417","418","419","420","421","422","423","424","425","426","427","428","429","430","431","432","433","434","435","436","437","438","439","440","441","442","443","444","445","446","447","448","449","450","451","452","453","454","455","456","457","458","459","460","461","462","463","464","465","466","467","468","469","470","471","472","473","474","475","476","477","478","479","480","481","482","483","484","485","486","487","488","489","490","491","492","493","494","495","496","497","498","499","500"]; setTimeout(function() {
    $('#slider-n-53').slider({
        stepping: 1, min: 0, max: 490, startValue: 0,
        change: function (e,ui) { var position = ui.value; if(values!=null) $('#slider-n-53-lbl').text(values[position]); interact(53, "sage.server.notebook.interact.update(53, \"n\", 1, sage.server.notebook.interact.standard_b64decode(\""+encode64(position)+"\"), globals());sage.server.notebook.interact.recompute(53)"); },
        slide: function(e,ui) { if(values!=null) $('#slider-n-53-lbl').text(values[ui.value]); }
    });
    if(values != null) $('#slider-n-53-lbl').text(values[$('#slider-n-53').slider('value')]);
    }, 1); })();</script></td></tr>
</table><div id='cell-interact-53'><?__SAGE__START>
        <table border=0 bgcolor='#white' width=100% height=100%>
        <tr><td bgcolor=white align=left valign=top><pre><?__SAGE__TEXT></pre></td></tr>
        <tr><td  align=left valign=top><?__SAGE__HTML></td></tr>
        </table><?__SAGE__END></div></td>
                 </tr></table></div>
                 </html>
}}}

{{{id=52|

///
}}}

{{{id=49|

///
}}}

{{{id=2|

///
}}}

<h2 style="text-align: center;">Statistics</h2>
<h3 style="text-align: center;"><span style="color: #333333;">create a natural, clean, native interface to stats functionality</span></h3>
<p><span style="color: #333333;"><br /></span></p>
<p style="text-align: center;"><span style="color: #333333;"><img src="r2.jpg" alt="" width="431" height="287" /><br /></span></p>

{{{id=57|

///
}}}

<p>&nbsp;</p>
<p style="text-align: center;"><img src="r.jpg" alt="" /></p>
<p><strong>Sage Includes Tons of Statistical Functionality:</strong></p>
<ul>
<li><strong>R -- </strong>R is included in Sage, along with the <strong>rpy</strong> C library interface and a pexpect interface to <strong>R</strong> (this was a major Sage-3.0 goal a year ago).</li>
<li><strong>scipy.stats -- </strong>Scipy's stats functionality is in Sage</li>
<li><strong>finance.TimeSeries</strong> -- highly <strong>optimized</strong> statistics (in the context of time series, but much more generally useful).</li>
</ul>
<p><strong>BUT</strong>, there is no nice friendly natural <em><strong>Sage</strong></em> interface to statistics.&nbsp; E.g.,</p>
<pre>sage: stat.mean([pi, e, sqrt(2)])
(pi + e + sqrt(2))/3
<br />sage: stats.standard_deviation(...)<br />sage: d = stats.data([pi, e, sqrt(2)])<br />sage: d.mean()<br />...<br /><br /><span style="font-family: times new roman,times;"><br /></span></pre>
<p>It is important that something like this be designed and implemented soon, since not having such functionality is a <em><strong>major</strong></em> <em><strong>shortcoming</strong></em> of Sage.&nbsp; Somebody needs to:</p>
<ol>
<li><strong>Math software stats survey: </strong>Make a survey of the statistics functionality and user interface (API) for each of MATLAB, Maple, and Mathematica.&nbsp; In particular, what special features or ideas do these CAS's have about how statistics and powerful mathematics software (including computer algebra) can be combined?</li>
<li><strong>Stats software survey:</strong> Similarly, survey the leading commercial statistics programs (e.g., SAS, Stata, etc.).&nbsp; What kind of data analysis features are most popular and important in those programs (e.g., which are the features their intro tutorials cover). </li>
<li><strong>Textbooks:</strong> Look at standard textbooks in statistics and see what functionality is needed to do those problems. </li>
<li><strong>Sage Enhancement Proposal (SEP):</strong> Design a rough interface for Sage (classes, functions, plotting capabilities, etc.), and post this for comment to Sage-devel.&nbsp; Always keep <em><strong>workflow</strong></em> in mind in the design (since I think that's how stats uses work -- they have data that goes in, pictures, analysis, then results that get plotted for papers, etc.) </li>
<li><strong>Prototype: </strong>Do a quick proof-of-concept implementation in pure Python, possibly calling out to R/scipy.stats/etc. for anything at all difficult to implement, so this can get done quickly. </li>
<li><strong>Feedback/optimize: </strong>Get the code from 4 into Sage, get feedback from users, and optimize.&nbsp; Optimization may include creating new Cython bindings to R, improving scipy.stats, etc. </li>
</ol>
<p>Andrew Hou -- a UW undergraduate -- will likely work on this project with me next quarter, but it would be great to have something going before then.</p>

{{{id=56|

///
}}}

{{{id=55|

///
}}}

{{{id=1|

///
}}}

<h2 style="text-align: center;">Combinatorics, Graph Theory, and Group Theory<br /></h2>
<h3 style="text-align: center;"><span style="color: #333333;">finish porting of MuPAD-combinat to Sage; remove Symmetrica from Sage; beyond Networkx;</span> implement code natively in Sage instead of GAP</h3>
<h2 style="text-align: center;"><br /></h2>

{{{id=62|

///
}}}

{{{id=61|

///
}}}

{{{id=60|

///
}}}

{{{id=18|

///
}}}

{{{id=63|

///
}}}

{{{id=17|

///
}}}

<h2 style="text-align: center;">Arithmetic</h2>
<h3 style="text-align: center;"><span style="color: #333333;"> many optimizations across the spectrum</span></h3>
<ul>
<li><span style="color: #333333;"><strong>Integer arithmetic -- eMPIRe: </strong>Despite protests from at least one Sage developer, Sage has switched to eMPIRe, which is a fork of the GMP project.&nbsp; I expect eMPIRe to soon be <strong><em>across the board</em></strong> superior to GMP in every imaginable way. <br /></span></li>
<li><span style="color: #333333;"><strong>Polynomial arithmtic in Z[x] -- FLINT</strong>: Sage has switched from NTL to FLINT for polynomial arithmetic in <strong>Z[</strong><em>x</em><strong>].</strong>&nbsp; Unfortunately, number fields still use NTL -- so number field elements have to be totally rewritten to use FLINT.&nbsp;&nbsp; Using FLINT provides important speedups. It's also getting used more an more in code directly as a library, since it is very easy to use from Cython.&nbsp; (E.g., massively easier than PARI because FLINT is well designed, and easier than NTL, since FLINT is in C instead of C++.)</span></li>
<li><span style="color: #333333;"><strong>Multivariate polynomial arithmetic -- libSingular</strong>:&nbsp;&nbsp; <ol>
<li> libSingular has been a nightmare from a porting perspective, but it seems the bugs are ironed out now.&nbsp; </li>
<li> They now support native arithmetic (and Groebner basis!) over <strong>Z</strong> (not just <strong>Q)</strong>, and we'll get that into Sage ASAP.&nbsp;</li>
<li> An exciting idea on the horizon is to make it so Singular can work with any Sage base ring (via PyObject*'s).&nbsp; This would remove the limitation on the size of the base field.&nbsp;&nbsp; </li>
</ol></span></li>
<li><span style="color: #333333;"><strong>Quaternion algebras -- </strong>Jon Bober and I completely rewrote them from scratch so that now basic arithmetic is <em><strong>very fast</strong></em>, over <strong>Q</strong> <em>and</em> number fields.</span></li>
<li><span style="color: #333333;"><strong>Symbolic expressions</strong> -- See pynac/calculus discussion above.&nbsp; This will thus soon be much more robust and faster.</span></li>
<li><span style="color: #333333;"> <strong>Matrices</strong> -- m4ri probably got faster.&nbsp; I implemented a <em><strong>lot</strong></em> of<em> small cases </em>over <strong>Z</strong> and <strong>Q </strong>by using PARI instead of some complicated asymptotically fast algorithms.&nbsp;&nbsp; Someone should do the same over $\mathbf{F}_p$.<br /></span></li>
</ul>

{{{id=67|

///
}}}

{{{id=66|

///
}}}

{{{id=65|

///
}}}

{{{id=16|

///
}}}

<h2 style="text-align: center;">Number Theory</h2>
<h3 style="text-align: center;"><span style="color: #333333;">quaternion algebras, quadratic forms, modular forms, descent on elliptic curves, Heegner points,&nbsp;<span class="typeset"><span class="scale"><span style="position: relative;"><span style="position: absolute; top: -0.048em; left: 0em;"> </span></span></span></span>L-functions</span></h3>

{{{id=15|

///
}}}

<ul>
<li><strong>Quaternion Algebras -- </strong>Sage now has an algorithm for computing the right ideal classes in an order in a quaternion algebra over $\mathbf{Q}$.&nbsp; The code is <em><strong>much much faster</strong></em> than Magma (or anything else available -- which is nothing).&nbsp;&nbsp;  <ol>
<li> Critical to algorithms for<strong> computing with modular forms</strong> and modular abelian varieties.&nbsp; </li>
<li>Mainly work of me, Gonzalo Tornaria, Alia Hamieh, and Jon Bober.&nbsp;&nbsp;</li>
<li> Sage does not have the analogue of this over <strong>number fields </strong>(L. Dembele's interest), which Magma has (due mainly to John Voight), but we need to implement it and make our implementation much faster than the one in Magma.&nbsp; Needed for Hilbert modular forms... </li>
</ol></li>
<li><strong>Quadratic forms: </strong>Jon Hanke's extensive quadratic forms code is now in Sage, and it provides a huge boost in functionality.&nbsp; This was years in the making.&nbsp; Jon is also rapidly improving the quality of this code.</li>
<li><strong>Modular forms</strong>: Craig Citro, <strong>David Loeffler</strong>, John Cremona, and I have all been putting a lot of effort into improving all the modular forms functionality in Sage.&nbsp;&nbsp; Especially David Loeffler!&nbsp; He's filled in gaps all over the place, pushed functionality in new directions, found very subtle bugs, etc.&nbsp;&nbsp; There is still much left to do, e.g., computing weight 1 forms, half integral weight forms, forms on other groups, emuch better code for working with congruence subgroups (Kurth's Farey symbols package).</li>
<li><strong>2-descent</strong>: Robert Miller just implemented 2-descent for rational elliptic curves with an isogeny.&nbsp; A huge surprise is that his code -- which combines all the latest idea and algorithms in <em><strong>novel ways</strong></em> seems to be blazingly fast, totally blowing away everything (e.g., both mwrank and Magma) by orders of magnitude.&nbsp; This will appear in Sage soon.</li>
<li><strong>3-descent</strong>: Robert is also working on implementing at least computation of $3$-Selmer groups.</li>
</ul>

{{{id=25|

///
}}}

{{{id=24|

///
}}}

{{{id=23|

///
}}}

{{{id=0|

///
}}}