Short introduction to the data.table library in R

[naupaka]

Scott and I put together an Rmd file with code and descriptions of several useful features of the data.table library to be used as a short ~3 minute lesson.

[ahmadia]

Thanks @naupaka and @sckott, I took a quick look and the overall format looks good.

@jdblischak, @karthik, do either of you want to mentor the review on this?

Note: There is still an open discussion on whether the bc repository is going to accept generated content (everything except the .Rmd file in the pull request, if I understand how knit works). I will leave it up to you if you would prefer to just commit the .Rmd file or the generated content as well until we come to a consensus, since the generated content is relatively light-weight.

[karthik]

I've dropped a few comments into the code above. Some general thoughts:

Overall the material is great for a 2-3 minutes discussion. But I see a few issues.

1. There is no organic place for this material at any of the three levels. It sticks out as an odd sidebar for a novice bootcamp, but too basic for an advanced one.
2. In January 2014 the dplyr package will be production ready and can handle data.frames, data.tables and sqlite and provide the same common syntax that we teach folks in our regular plyr section. So people who understand plyr operations will then benefit from the power and speed of data.table without learning the arcane syntax. So by then, the only folks who will need to learn the ins and outs of data.table will be advanced users with special use cases that can't be accomplished by the simpler plyr format.

My recommendation: This is ok to merge as a standalone lesson but I don't see much use for it in the immediate term. It could be brought up in case some students bring up speed or "big" data issues.
But in the long term it would be great to see this fleshed out as a complete data.table tutorial that could perhaps be a topic for an advanced R bootcamp. The data munging/manipulation section we have in the works for the intermediate section could be replaced by this for the advanced users. It would be also great to have more than simulated data as examples.

So @ahmadia I'll leave it up to you to merge after you hear back from @jdblischak

[jdblischak]

Thanks for your contribution! Here are my thoughts:

• I think this belongs in the intermediate lessons. I'd suggest putting it in a folder such as r/intermediate/misc/data.table/.
• Use head more often. This will make it easier to follow from the html version of the lesson.
• Since this would need to be incorporated as an add-on, you may want to include instructions such as prerequisite knowledge required, duration time (like you have in the PR message, though 3 minutes seems short to me), and learning objectives. Perhaps you could put this information in a README.md that resides in the same directory as your lesson.
• An exercise or two would be useful.
• I don't think anything beyond the RMarkdown file is necessary in the main repo. Generated content can be committed to bootcamp-specific repos. Doesn't matter to me though.
• I personally do not like having text lines that cannot be read on GitHub. I wish RStudio had an option like fill-paragraph (alt-q) in Emacs to automatically adjust paragraphs, but I haven't been able to find a convenient method. However I don't know if anyone else feels the same, so I leave that to your judgement.

[jdblischak]

suggestion: "often-completed" -> "common"
typo: "Let" -> "Let's"

[ethanwhite]

I personally do not like having text lines that cannot be read on GitHub. I wish RStudio had an option like fill-paragraph (alt-q) in Emacs to automatically adjust paragraphs, but I haven't been able to find a convenient method. However I don't know if anyone else feels the same, so I leave that to your judgement.

GitHub is actually supposed to be sof

[wking]

On Tue, Dec 10, 2013 at 03:29:15PM -0800, Ethan White wrote:

GitHub is actually supposed to be sof

…t wrapping? I thought so too, but maybe .Rmd doesn't count as
“prose” 1?

[ethanwhite]

Oops, yes, thanks. It works on the .Rmd on my phone, but not in either Firefox or Chrome on my laptop.

[naupaka]

Thanks all for the helpful comments! I will get to the other suggested changes in the next couple days.

[gavinsimpson]

It is a package, not a library. The latter is a collection of the former.

[naupaka]

Mybad, thanks for catching that. Will be fixed in the next commit.

[ethanwhite]

GitHub is supposed to be soft-wrapping prose

I have a PR in to Linguist that would fix this for Rmd files (github-linguist/linguist#831). However there is a test currently failing on Ruby 1.8.7 that I don't know enough to debug. If someone with more Ruby experience (@emhart ?) has a chance to take a look fixing this should increase the chances that this gets implemented quickly.

[ahmadia]

@naupaka - Is this ready for review again?

[naupaka]

Not yet, Scott and I still have stuff to fix. Been traveling.

[ahmadia]

No problem, just making sure you weren't silently waiting on me :)

[emhart]

@ethanwhite I checked this out, but I see that they merged the request and that your fork was building fine. Did you resolve the problem?

[karthik]

@emhart yes, resolved. That discussion was just referenced here and should continue on the original thread if there is anything further.

[gvwilson]

Is this one ready to merge?

[ahmadia]

It's waiting on @naupaka :). @naupaka - can we help with anything?

[sckott]

I'll take a look and get things fixed, hopefully today. I'm partnering with @naupaka by the wway :)

[sckott]

Okay all, I've fixed up the data.table lesson. @naupaka Anything else on your end?

[naupaka]

Let me take a quick look in the next few hours and I'll let you know if I have anything else to add. Thanks!

On Jan 21, 2014, at 5:52 PM, Scott Chamberlain [email protected] wrote:

Okay all, I've fixed up the data.table lesson. @naupaka Anything else on your end?

—
Reply to this email directly or view it on GitHub.

[naupaka]

ok @sckott and @ahmadia looks good to me.