# SageMathCloud Blog

## SageMathCloud is Now CoCalc

Hal Snyder • • cocalc

As of May 20, 2017, SageMathCloud is being renamed to CoCalc, for Collaborative Calculation in the Cloud.

## A Natural Evolution

SageMathCloud was introduced in April 2013 as a hosted platform for the SageMath software system. It saved thousands of users the time and effort of building, installing, and updating SageMath locally. SageMathCloud provided a single consistent platform for improving the range and quality of services offered.

Over time, we began adding new features, mostly for our own needs: a course manager to teach students SageMath, chat to coordinate within the platform, a $\LaTeX$ editor to write documents, and Jupyter notebooks to support fellow instructors. In addition to conventional backups, we added snapshots and time travel. We added support for numerous languages, including Python 3 as well as Python 2, R (including R markdown), the complete Anaconda distribution, Octave, Julia, C, C++, Haskell, and Scala. We included major libraries and packages for these languages, and update them regularly.

We became a platform that enables students, teachers, and researchers to collaborate using a broad selection of programming languages and development tools – not just SageMath.

While SageMathCloud suggests exactly what we started out as, we have certainly outgrown it. The new name, CoCalc, reflects the evolution into a general purpose, cloud-based calculation and computation platform. Our recent move to rewrite the Jupyter notebook to be more integrated and fully supported is an example of how our mentality has shifted.

## The CoCalc Experience

Newcomers can explore CoCalc services from different approaches:

• Projects. In CoCalc, work happens inside projects. A project is a personal workspace where you can group related files.

• Collaboration. By inviting collaboraters into your project, you allow them to see and chat about your work, and even edit it with you in real time.

• Teaching. With Cocalc’s teaching tools, you can setup courses, create handouts with mathematical formulas, and manage homework and grading.

• Authoring. CoCalc has extensive support for $\LaTeX$, in document editing as well as all interactive notebooks and chat.

## What Should I Do If I’m Already an SMC Subscriber?

When you visit https://cloud.sagemath.com after the change, you’ll be immediately redirected to https://cocalc.com. You’ll then have to sign in again the first time. That will set a cookie, and after that everything will be as normal. Anyone following a link into the old name, e.g. https://cloud.sagemath.com/some_path..., will be redirected to the same path under the new name, https://cocalc.com/some_path....

You will notice our documentation changing to use the CoCalc product name and the CoCalc.com domain name.

We have created a new CoCalc discussion group in Google Groups here and we will automatically subscribe all current sage-cloud subscribers to it.

## What Stays the Same

The company behind CoCalc will remain SageMath, Inc. Our staff email addresses will remain in the sagemath.com domain.

More importantly, we remain committed as ever to improving the experience of scientific and exploratory computing for students, teachers, and researchers around the world.

## Try CoCalc Today

If you’re not already a subscriber and wonder what all the fuss is about, now is a great time to get acquainted. Start CoCalc for free today.

## New SMC Jupyter Client Released

Hal Snyder • • jupyter

Today SageMath, Inc. is pleased to announce the release of a completely new Jupyter notebook implementation, optimized for SageMathCloud, our cloud-based, collaborative environment.

This new implementation was motivated by the need for better collaboration and browser/server synchronization. Rewriting also offered an opportunity to integrate previously-unavailable features of SMC into Jupyter notebooks. However, throughout this process we minimized interface discrepancies between old and new versions so that switching is as seamless as possible.

Here’s an overview of the most significant enhancements in today’s release:

## Collaboration

### Simultaneous Editors

Simultaneous editing by multiple people is now fully supported, including multiple cursors and document-wide, user-specific undo and redo.

User One and User Two editing a notebook at the same time.

### Chat

Each notebook now has a chat sidebar. As always, chat supports markdown, $\LaTeX$, and video chat.

Text chat with markdown shown in panel to right of notebook body.

## Server-side enhancements

### TimeTravel

Jupyter includes TimeTravel just like our other editors. It shows detailed history of all changes to a notebook, and the author of each change.

Time Travel view of previous notebook as seen by User Two
showing revision 93 (of 95) created by User One.

### Output

• Sophisticated handling of large output: throttling, windowing, back-end buffering.
• Background capture of execution output, even if no user has the notebook open in their browser.

## User interface

• Cleaner, more modern look with buttons and menus that better reflect state.
• Better mobile web support.
• Uniform font sizing.
• Code folding.
• Vim, Emacs, and Sublime keybindings, and color schemes (in account settings).
• Export notebook file to LaTeX using “File > Download as… > LaTeX (.tex)…”.

Sample notebook about to be exported.

Exporting notebook to Latex and opening the .tex file in SMC.

Converted ipynb file opened in LaTeX editor.
• A purely client-side notebook viewer for easily sharing your work publicly.
• Raw file edit mode: synchronized editing of underlying JSON ipynb file. See resulting changes to the rendered notebook in real time.

## RethinkDB versus PostgreSQL: my personal experience

William Stein (wstein@sagemath.com) •

## Introduction

Initially motivated by the shutdown of the RethinkDB company, and the licensing situation with RethinkDB (a blocker for certain parts of my business), I worked very hard for two months to completely rewrite the realtime and database components of SageMathCloud (SMC) to use PostgreSQL instead of RethinkDB, initially motivated by this discussion in Hacker news. I battled with and used RethinkDB heavily since May 2015, and I’ve used PostgreSQL heavily as well, with production data, rewriting all the same queries in both systems, so I’m in a good position to compare them for my use case (the site SMC).

This is my story. It’s a personal comparison, with NO BENCHMARKS or hard data you could reproduce. It’s what I would tell you if we were talking by the water cooler.

Summary:

• I’m very happy with the rewrite.
• Everything is an order of magnitude more efficient using PostgreSQL than it was with RethinkDB.
• It is much easier to do exploratory queries of our data using PostgreSQL than it was with RethinkDB. PostgreSQL is much more expressive than ReQL, has a massive number of built-in functions, so we are making much better use of our data. With RethinkDB, often we just ended up greping through the latest database dump.
• PostgreSQL is “statically typed”, whereas RethinkDB had no type or schema enforcement at all; explicit clear typing improved the quality and robustness of our application.
• We are saving $800 month (!), due to reduced CPU and disk space requirements. • I had no clue that RethinkDB would be Apache licensed in February 2017. ## SMC for Collaborative LaTeX Editing Hal Snyder • • latex SageMathCloud (SMC) is the most powerful online$\LaTeXcollaboration software available today. SMC offers the full complement of features expected from online services today for professionals and students, on a par with and exceeding other leading products, such as Overleaf and ShareLaTeX. In addition, SMC offers a complete environment for teaching, research, and exploratory computing with the same rich feature set. Here’s an overview of key SMC features to back up the claims of the previous paragraph. ## Inline LaTeX Errors haraldschilly • • latex Small update of our LaTeX editor. Now it shows LaTeX errors inline, which should make it easier to fix these problems. The parser reading the LaTeX log was also improved to treat consecutive errors separately. Error location mapping also works for .Rnw knitr source files. ## Wishbone haraldschilly • • r Do you want to run Wishbone in your SMC project? Wishbone is an algorithm to align single cells from differentiation systems with bifurcating branches. Wishbone has been designed to work with multidimensional single cell data from diverse technologies such as Mass cytometry and single cell RNA-seq. First, open a terminal file and run the following lines to install it locally inside your project: pip3 install --user Cython pip3 install --user git+https://github.com/dominiek/python-bhtsne.git pip3 install --user git+https://github.com/jacoblevine/phenograph.git pip3 install --user git+https://github.com/ManuSetty/wishbone.git  Then, open a new Jupyter notebook, switch to the “Python 3” kernel, and run examples from their documentation. (In order to access GitHub or PyPi, your project needs “internet access”) ## R updated haraldschilly • • r Dear R users! We’ve switched our default R to be the “official” one from https://www.r-project.org/. Therefore the default R version has been updated from version 3.2.4 to the most recent one 3.3.2. This is a quite significant update from the one in SageMath, hence this notice. If for some reason you still need to work with the older version of R in SageMath, do this: 1. In the command-line use R-sage instead of R and Rscript-sage instead of Rscript. 2. In Jupyter notebooks there are two R kernels: the newer “R (R-Project)” and “R (SageMath)” for the version shipped with SageMath. Regarding libraries, we installed all the ones we know about and many more. In total, there are 557 R libraries (1.3GB) available. Is something that you need still missing? Email help@sagemath.com. ## using SMC with python & data science MOOC halsnyder • • python, mooc, and datascience Here are a couple tips based on my experience using SMC to complete the Coursera course, Introduction to Data Science in Python. This course is the first installment of a new 5-part Applied Data Science with Python Specialization from the University of Michigan. Examples and study assignments for the course are offered as Python3 Jupyter notebooks, i.e. .ipynb files. Students may use Coursera-hosted jupyter notebooks or any other platform that allows them to run the code. Homework is submitted by uploading a .ipynb file for each programming assignment. The following might be helpful for students taking the course and using SMC: ### 1. Set Jupyter kernel. After uploading a course .ipynb file, change the kernel from Python3 to Anaconda (Python3) as shown below. This will prevent errors such as AttributeError: ‘Index’ object has no attribute ‘str’ due to different versions of pandas. ### 2. Convert Jupyter notebooks to Sage worksheets. If you would rather code in a Sage worksheet than a Jupyter notebook, use the SMC script smc-ipynb2sagews to convert the files. Open a terminal file, for example mooc.term, and enter the following commands  cp Assignment\ 2.ipynb assgn2.ipynb
$smc-ipynb2sagews assgn2.ipynb /usr/local/bin/smc-ipynb2sagews: Creating SageMathCloud worksheet 'assgn2.sagews'$ open assgn2.sagews


## NLTK text corpus

haraldschilly • • python

The full 2.4gb NLTK text corpus is now available. You can for example run this in our SageMath or the Anaconda Python environment:

from nltk.corpus import brown
w = brown.words()
len(list(w))


which gives

1161192


## Install Jupyter's nbextensions configurator

haraldschilly • • jupyter

You can install it in your own project. For that, you need internet access enabled or somehow upload the code into your project. Then, install it like this in a terminal (create a new file terminal.term)

 pip install --user --no-deps jupyter_nbextensions_configurator
jupyter nbextensions_configurator enable --user


and restart the Jupyter server in SMC

smc-jupyter restart


Then, in order to see the configurator, you have to open an ipynb file. Click on the the “About” button in the top right click on the link there to open the version of jupyter without the synchronization. There, either go to the main page or the one dedicated for the nbextensions. The URL looks like this:

https://cloud.sagemath.com/<your_project_id>/port/jupyter/nbextensions