Google Summer of Code 2020: The Final Report

This summer, I participated in the Google Summer of Code (GSoC) 2020. This post summarizes my work as the final report of GSoC.

My project page on the official site: Brush up RBS and related tools for practical Rails apps.

What I did (TL;DR)

Background

As written in my project's description, RBS is about to be released as a part of Ruby 3.0. RBS is not only a type-annotation language name but also a tool name to manipulate RBS languages. RBS is actively developed by Soutaro Matsumoto, a lead Ruby engineer at Square. There is a nice post to introduce RBS written by him: The State of Ruby 3 Typing.

Gradual typing with RBS allows Rubyists to enjoy the benefits of static analysis. It would be welcomed by Ruby users who also love recent Python or TypeScript.

However, RBS is still a developing technology and its immediate applicability is somewhat unknown. In particular, the type-checking of practical Rails apps is important because Rails is the major usage of Ruby language.

There are some other characters related to this story: RBS Rails and Steep. RBS Rails, maintained by Masataka Pocke Kuwabara, integrates RBS and Rails with auto-generating Rails-related RBS signatures (*.rbs). Steep, maintained by Soutaro Matsumoto the same as RBS, performs type checking of Ruby implementation (*.rb) with RBS signatures.

The actors are all in place: RBS, RBS Rails, and Steep. In this project, I tried to verify and improve the practicality of these three tools.

Details

My work this summer can be categorized into three parts.

  • Application of RBS tools to practical Rails apps
  • Various improvements of RBS tools
  • Implementation of new RBS subcommand rbs subtract

Application of RBS tools to practical Rails apps

I applied three tools to two practical Rails apps.

Ruby CI checks and shows test results of each commit of the central Ruby repository as a web site. It was chosen because of its small size and relationship to us.

Mastodon is a self-hosting microblog service like Twitter. It was chosen because of its larger size and popularity as same as other real services.

I wrote *.rbs file for these two apps, but the common parts as a Rails app are automatically generated RBS Rails. While continuing to write the RBS, I used Steep to perform static analysis.

They mostly worked well and usable even for large model classes. Minor issues were found along the way, but they were all either corrected or reported as written in the next "improvements" clause and did not affect the overall work.

I believe that these have shown that the RBS and related existing tools are enough effective and practical for developing Rails apps.

However, the tool was mainly applied only to the model classes of the code, and we only worked on one model for Mastodon, which has a large codebase. Application to controllers, views, helpers, etc., will be future work.

Implementation bugs

Most of the implementation classes were judged as correct by the Steep, a type checker, but it found some bugs in two apps.

Ruby CI had a bug that yields a 500 error. It was caused by a lack of  nil-checking and was fixed by my PR.

Mastodon's User model has a reset_password! method, but it seems to dead code because it is never called in other files including gems that depended. Steep detected that a calling of super will cause NoMethodError. It was true that Devise removed the method 4 years ago, but  further careful checking of the code and creating a PR for removal is a future task.

Various improvements of RBS tools

While I applied the tools to Rails apps, I found various problems. All of them are reported to the upstream repositories.   

RBS

RBS Rails

Steep

All reports were found based on experiments of application for real Rails apps, and I think these improvements made RBS more practical.

Implementation of new RBS subcommand rbs subtract

At the end of the project, I tried to implement a new RBS CLI feature called rbs subtract. This is not completed and merged yet, but this feature can improve the workflow of Ruby development with RBS.

Why needed?

Writing an RBS signature file from scratch is hard. That's why RBS can generate prototype signature files as a command rbs prototype.

Implementation -> Prototype RBS

But this command is intended to generate files at first. There is no ability to detect the difference in files.

And while they make it easier to write, the auto-generation is not perfect always, so manual modifications will still occur.

Implementation -> Prototype RBS -> Modified RBS

On the other hand, the development of the implementation files also proceeds. This requires new changes to the RBS, but again, the RBS can generate a prototype automatically.

Implementation -> Prototype RBS -> Modified RBS (B)
          　　 \-> New Implementation -> New Prototype RBS (A)

This is where the problem arises. The developer can generate new RBS A, but this needs to be modified by hand, as I wrote earlier. On the other hand, B, which has been modified by hand, is already outdated.

This means that the developer will be forced to manually merge A and B at the same time as rewriting the implementation.

However, A often has more classes and methods than B. Ideally, this increased differential should be the only thing that needs to be manually corrected. And the developers will want to use the B for the existing parts.

So we need A - B. This is where the rbs subtract comes in.

The rbs subtract is a command to subtract a signature literally. If the argument is the file equivalent of A, it outputs the RBS as the difference between methods, constants, etc. that are not in the existing signature, but only in A. The actual flow of work will look like this.

  1. Generate the first prototype and modify it to fill the correct types
    $ rbs prototype rb app.rb >sig/app.rbs
  2. Update implementation file app.rb
  3. Generate an incomplete but new prototype file
    $ rbs prototype rb app.rb > app-new.rbs
  4. Subtract to get differences
    $ rbs subtract app-new.rbs >> sig/app.rbs
  5. Now you should only fix the differences added to sig/app.rbs

Automatically generating only the differences added to the implementation lowers the cost of writing RBS signatures while development. Conversely, if an implementation is decreased, this is a future issue, but the real harm is small in this case, as the presence of extra signatures is unlikely to result in an unchecked or incorrectly checked implementation.

Implementation details

This is an ongoing working branch of rbs subtract. I added RBS::AST::DuplicationFilter to eliminate duplication between old and new signatures in the AST level. Since it is a new CLI subcommand, cli.rb was changed to add the run_subtract method that using DuplicationFilter. 

The most difficult part is to cover all patterns of AST, but it was mostly done.

I will continue coding to realize this feature.

Conclusion

I have applied RBS tools to practical Rails apps such as Ruby CI and Mastodon. Not only did I demonstrate the practicality of RBS, but we also found real bugs on the implementation side.

I improved and fixed various problems of RBS and related tools that based on the above experiment. I think that the quality of the tools was raised to a practical level.

I also implemented most of a new RBS subcommand rbs subtract. This can streamline your development workflow. Completing and merging the implementation is a future task.

Acknowledgment

First of all, I would like to thank my mentor, Yusuke Endoh. It was reassuring to work with him, who has a deep knowledge of Ruby and is also an implementer of a type profiler.

Thanks also to Soutaro Matsumoto and Masataka Pocke Kuwabara for their continuous support. It was really helpful for me to have weekly Zoom meetings with the four of us, especially to get advice from the RBS, Steep, and RBS Rails maintainers. I'd also like to thank other Ruby committers and Rubyists for their opinions, including Koichi Sasada.

I would like to thank everyone involved in GSoC as a Ruby Organization, especially Hiren Mistry, who organized the whole thing. We had international Zoom meetings in English once a week, and I received a lot of positive advice and encouragement from the occasional guest. I was nervous at first because my English conversation skill is not so good, but thanks to the thoughtfulness of Hiren and other mentors, the atmosphere was kept cheerful.

Finally, I'd like to thank Google for organizing the Summer of Code project and especially the staff who involved in it. I understand that it has been difficult to keep communicating with students scattered around the world with only a few full-time members while running multiple projects at the same time. I am grateful to continue to run a project like GSoC, which is also a social contribution.

Comments

Post a Comment

Popular posts from this blog

My current status of Ruby Summer of Code

My project, Decimal , is going well. I made several milestones for the midterm evaluation, and all of them are almost completely done. Note that tarball/gem of the latest decimal's doesn't reflect the work in this summer.  If you want to get the latest features now, please checkout it from its subversion repository of trunk.  (I'll solve this issue soon in upcoming release.) I'll list the milestones up: Implementing Web pages First of all, I made a simple web page.   http://decimal.rubyforge.org/ It might look too simple, but I think this is succinct.  At least it looks even better than the previous. Now you can find out: How to download / install Latest API documents Existence of mailing lists Coding examples Mathematical functions implemented This is the easiest point to understand of current works. I implemented 24 math functions as Decimal::Math. function () , with test cases of course.  Most of them are not in BigDecimal but in Float a
Read more