Source maps

[guybedford]

This is not an urgent request - just some consideration as to how this could be implemented.

CoffeeScript Redux supports source maps generation (Ryan Florence has a good write up here http://ryanflorence.com/2012/coffeescript-source-maps/)

It does support the browser environment as well, but needs the source-maps dependency to generate the source map. The source-map module is fully AMD compatible in the browser which is a big bonus.

So it may take some work to get CoffeeScriptRedux to work with source map generation in the browser within AMD, but it looks like it may be possible, perhaps with a CoffeeScriptRedux pull request or two.

Then, the question is how to populate the source map within eval'd code generated from CoffeeScript. Looking through the spec, I can't see if this is straightforward or not.

One way might be to set the source map as a data-url within the sourceMap comment. This is worth testing to see if it is supported.

Getting this source maps support is quite crucial to long-term buy in for require-cs. I'm very much a fan of the dynamic compilation that AMD implies for a development environment. If we miss crucial features like this, static compilation may win by adoption though.

[jrburke]

Yeah, I don't quite understand how a multi-stage source map (CS -> JS -> minified JS) would work. I will likely need someone to research this and figure it out, and do pull requests. The source map support in 2.1.3 is what I would consider experimental, and it really just came along for the ride with the uglify2 work.

So while I am open to getting this work, someone else will need to do the heavy lifting as I have not had a personal use case that needed source maps.

[guybedford]

The hard part is providing a source map to eval'ed code I think - I will do some tests and see if I can get this to work with data-uris.

In terms of the bigger picture, I'm now using CoffeeScript quite heavily in a dev environment with cs! requires all over the place. I'd be interested to hear if you think this is a good approach, or if it would be more sensible to have a compiler running as a process in the background and then using js requires on the compiled files?

[jrburke]

For the evaled code, maybe the answer there is @sourceURL that also as an @sourcemap thing.

Supporting transpiled languages is very important for a module system, something that should work without needing a precompile step. So I'm glad to have the cs plugin getting use, and I do want to work out any wrinkles with its use (requirejs/requirejs#497 for example), if for no other reason to work out the right flow for ES modules later. So, I am still committed to making it work. Source maps in general are just a hard sell for me, as they introduce a lot of complexity, and they still seem new (I don't hear much about the multi-stage maps for example).

If a precompiled cs step can work out the sourceMap translation from CS -> JS -> minified JS, then maybe we can use that knowledge to figure out the pathway here.

[guybedford]

I tested the eval method as described above in a test repo here - https://github.com/guybedford/cs-source-maps (to see what is going on see https://raw.github.com/guybedford/cs-source-maps/master/test-eval.html)

I noticed that webkit seems to support the sourceMapperURL with a data-uri, only the original source isn't displayed. I've filed a bug here https://bugs.webkit.org/show_bug.cgi?id=63940#c13.

If this is sorted out, then the entire method I described above will work perfectly for debugging in a development environment. It is specifically getting full source mapping in the development environment that is the focus here.

[guybedford]

I filed a proper bug here - https://bugs.webkit.org/show_bug.cgi?id=107939

[guybedford]

A patch has been submitted here - https://bugs.webkit.org/show_bug.cgi?id=107939

This means that we just need to do:

• Use CoffeeScript Redux for CoffeeScript compilation
• Run through the process of getting it to generate a source map in the browser with the source maps module
• Re-encode the source map as a data-uri to add to the @SourceMappingURL property comment in the eval code, along with the @SourceURL comment

Then with the above patch in webkit, full debugging would be possible during development (note it is specifically the development environment I am focusing on).

Firefox still doesn't support the @SourceURL comment in evalled code as far as I can tell (https://bugzilla.mozilla.org/show_bug.cgi?id=833744). So once that is through it will be again necessary to check how it works with the test repo at https://github.com/guybedford/cs-source-maps (test-eval.html) is the file. If debugging isn't supported yet it would be worth chasing up the issues there as well.

[jrburke]

Not sure if this helps, but meant to mention this earlier.

This tutorial:

http://net.tutsplus.com/tutorials/tools-and-tips/source-maps-101/

mentions that uglifyjs supports an --in-source-map property that could take a cs-to-js source map and use that as part of generating the minified source map. So maybe just have a way to tell the optimizer to look for [file name].coffee.js.map and if there feed to the --in-source-map property.

[tomjakubowski]

the standard implementation of CS supports source maps as of 1.6.0, by the way

http://coffeescript.org/#source-maps

[esamattis]

They got source maps working with the <script type="text/coffeescript"> tag.

jashkenas/coffeescript#2780
https://github.com/hden/coffee-script/blob/c24e957f1714e67240447deb643dff6b59b56ff6/src/browser.coffee#L15

I think we could do something similar here too.

[guybedford]

Amazing, that looks like it would adapt perfectly here. Will check it out when I have a moment too.

[esamattis]

Also coffeeify from Browserify v2 can be used as a reference: https://github.com/substack/coffeeify/commit/9fb0ab24e923e66039304c1b5b7a84ec14a1ed86

[jrburke]

I have an in-progress branch for source map support that goes from separate .js files to concatenated files, and then to uglify2 minification here:
https://github.com/jrburke/r.js/tree/sourcemap

I'm still working out the file naming, in particular so that the uglified maps point back to non-uglified code, so that tree is probably not interesting to the outside world yet. I also want to allow transpiler plugins to work. However, I have not worked out how a loader plugin would trigger its source map generation and saving. r.js allows building to a string output, so the plugin should not write to disk directly, but perhaps pass the data back somehow in the "write" loader plugin method. For writeFile, it probably should have a way to write directly to disk.

So I could use some help working out those mechanics. I have been very busy with day job stuff, but I do hope to get back to finishing up the sourcemap tree later this week, and if I get really lucky doing a 2.1.6 release with its changes before next week. Depending on how much churn it is for loader plugins, that capability may have to wait for the release after 2.1.6, but I would like to try for 2.1.6 if we can cleanly sort out the details.

[guybedford]

The source map support in https://github.com/hden/coffee-script/blob/c24e957f1714e67240447deb643dff6b59b56ff6/src/browser.coffee#L15 seems ideal for loader plugin support. It uses a generated data-uri to describe the source map, so that it doesn't need to be saved on the server, which is what I was initially attempting above. Between the sourcemap branch, and a loader addon like this, that would cover the production and development environment source maps respectively.

I'm just getting into my open source time for this week... have a few other things to do, but will give the dev environment loader source maps an attempt if I can.

[guybedford]

I attempted the source maps today and got quite far. CoffeeScript is generating the map, and the original CoffeeScript is displaying in the Chrome debugger.

But as soon as I change the path to a full URL everything falls over, and debugging doesn't work!

I've tried quite a few permutations of options, but can't get this to work, so either I am misreading the docs or Chrome has a few issues to work through yet. Any help would be really appreciated here.

Code is here - https://github.com/guybedford/require-cs/blob/master/cs.js#L147

[jrburke]

I thought if sourceMappingURL was in play then sourceURL would not be needed.

By using a data URL that works around the need for exterior files, but I wonder if this would be desirable since it means the compiled JS would include a big source map in the source. There would be better performance by allowing the source map to be saved as a separate file.

Although maybe this approach can be used, and r.js would scan for @sourceMappingURL and if a data URL then pull it out and save as a separate file.