Scala: The language of agility?

I’ve been thinking about writing this post for a while now, and this post bidding farewell to Scala has prompted me to finally do it. In particular, this snippet from the first paragraph:

But as soon as I started working in teams writing Scala, your immense syntax started drowning me. At first, I took it as a compliment that you tried to please me by offering me to work the way I liked. But then I noticed that it wasn’t something you did for me in particular. Instead, you try to be everybody’s darling by offering every software development paradigm known to man.

Matthias raises a valid and oft-cited point here. Scala has no “Right Way” or opinion on how to do things. There is no prescriptive guidance offered. You can work with Scala as a better Java much like Groovy, or you can use it as Haskell-lite on the JVM with Scalaz. As Neal Ford once described Scala, it is “a vast parking lot from here to functional programming, where you can go anywhere or any distance and stop to set up camp where you like.” (Not really a quote because this is from memory, which is an unreliable source). I am in 100% agreement with Matthias and Neal.

However, I don’t agree that this is necessarily a bad thing. There are many merits to having a language (or any tooling for that matter) with a stricter paradigm for having a “Right Way” to do things. For instance, it allows less friction for developers moving among teams with similar stacks. It can also allow stronger more powerful invariants of your applications. But are there merits on the other side of the fence?

I think so. I’m not a big fan of prescriptive guidance. While such things can be good starting places (like Scrum for organizations which have been long-sentenced to waterfall), they don’t pan out well over time. Eventually the team sees that its needs are better met by breaking the rules. In fact, my team has recently cast aside scrum in favor of a continuous-flow/kanban approach to our development. I’m also rather fond of my tooling allowing the same freedom.

My team is able to decide what our approach is. Are we just a better Java? Nah, I think we’re further along than that. Are we Haskell-lite? Nope, Scalaz is not in the dependency list. Frankly, where we are today is a big stretch from where we were. There is no way I could have sold this team on Haskell or even Clojure. They weren’t ready for unadulterated functional programming. I’ve had to ease everyone (myself included) into this paradigm. As we get better, our standards change and the language is able to adapt to that.

Another good example of this is my beloved Lift. Even though former BDFL David Pollak recently lamented not having a “Right Way” to do things in Lift like Ruby on Rails, I think Lift got it right. This is one of the very things that attracted me to Lift from my previous love in web, Grails. While working with Grails, I was always forced to solve problems the way they want me to. Unfortunately for that approach, they wanted me to solve problems I didn’t have. In my past life in telecom, the management application I worked on was not focused on a database as the central source of information. Instead, the source of truth was a live network where the database was really just a cache. Having an MVC approach where all of the models autowired themselves to a database didn’t make any sense. With Lift, I’m able to write an application that is backed by whatever I want. It can have no database at all. It’s just a tool that solves problems related to serving HTML, resources, etc.

I’m not attempting to dismiss the objections of those who point out this characteristic of Scala. It is real, and it is a problem for some teams/people. I would just like to highlight some of the good things which can arise out of the approach taken by Scala. In agile, decisions are always left up to the team. I feel that we made the right choice with Scala for this reason. Rather than demand we mold our domain problem to the tooling, this malleable language molds itself to our domain.

Leave a reply below, or send me a tweet.

Tagged with: , , , ,
Posted in Software Development

The JVM Bytes: Pilot Post

Thanks to a recent talk by Heroku’s Joe Kutner, I got inspired to learn how to program on the JVM. I’m sure you’re thinking I’ve been doing just that for years with Java, Groovy, and Scala. However, I’ve yet to write straight to bytecode. Joe’s aforementioned talk covered the basics of how the JVM works under the hood of our languages. He introduced the JVM architecture and tools for reading and writing our own bytecode. I have been writing a lisp compiler lately to explore this subject, and I’ve decided it is time to kick off a blog series on the topic of JVM bytecode.

What is bytecode? It’s the *.class files emitted by your compiler. It is a low-level binary format much like native machine code. The difference being that the JVM itself is another layer of software between your bytecode and the metal of your machine. The point to emphasize here is in order to view the bytecode directly, you’ve gotta use a hex editor. Well, that’s quite difficult to understand of course. What we want is something that is a little higher-level than the bytes, much like assembly language. There is no assembly for the JVM, tho. The languages to straight to the bytes. Fortunately the JDK includes javap which will print out the bytecode of a class file in a human-readable format. Invoke javap -v -c <classname> in the root directory of your class files to view it.

For example, compile a typical Hello World program in java then view the javap output. This java class…

public class Hello {
    public static void main(String[] args) {
        System.out.println("Hello Java!");
    }
}

…produces the following javap output…

>javap -v -c Hello
Compiled from "Hello.java"
public class Hello extends java.lang.Object
  SourceFile: "Hello.java"
  minor version: 0
  major version: 50
  Constant pool:
const #1 = Method       #6.#15; //  java/lang/Object."<init>":()V
const #2 = Field        #16.#17;        //  java/lang/System.out:Ljava/io/PrintStream;
const #3 = String       #18;    //  Hello Java!
const #4 = Method       #19.#20;        //  java/io/PrintStream.println:(Ljava/lang/String;)V
const #5 = class        #21;    //  Hello
const #6 = class        #22;    //  java/lang/Object
const #7 = Asciz        <init>;
const #8 = Asciz        ()V;
const #9 = Asciz        Code;
const #10 = Asciz       LineNumberTable;
const #11 = Asciz       main;
const #12 = Asciz       ([Ljava/lang/String;)V;
const #13 = Asciz       SourceFile;
const #14 = Asciz       Hello.java;
const #15 = NameAndType #7:#8;//  "<init>":()V
const #16 = class       #23;    //  java/lang/System
const #17 = NameAndType #24:#25;//  out:Ljava/io/PrintStream;
const #18 = Asciz       Hello Java!;
const #19 = class       #26;    //  java/io/PrintStream
const #20 = NameAndType #27:#28;//  println:(Ljava/lang/String;)V
const #21 = Asciz       Hello;
const #22 = Asciz       java/lang/Object;
const #23 = Asciz       java/lang/System;
const #24 = Asciz       out;
const #25 = Asciz       Ljava/io/PrintStream;;
const #26 = Asciz       java/io/PrintStream;
const #27 = Asciz       println;
const #28 = Asciz       (Ljava/lang/String;)V;

{
public Hello();
  Code:
   Stack=1, Locals=1, Args_size=1
   0:   aload_0
   1:   invokespecial   #1; //Method java/lang/Object."<init>":()V
   4:   return
  LineNumberTable:
   line 1: 0

public static void main(java.lang.String[]);
  Code:
   Stack=2, Locals=1, Args_size=1
   0:   getstatic       #2; //Field java/lang/System.out:Ljava/io/PrintStream;
   3:   ldc     #3; //String Hello Java!
   5:   invokevirtual   #4; //Method java/io/PrintStream.println:(Ljava/lang/String;)V
   8:   return
  LineNumberTable:
   line 3: 0
   line 4: 8
}

Ok, so that is definitely a lot of information. I’m not going to attempt digesting all of this information in this blog post. Frankly, I don’t understand it all, and we can begin making progress with a few observations.

First take note of the constant pool. If you have a fair amount of experience on the JVM, there is a good chance you are already aware that any String literals in your code are stored as constants for the entirety of the program. We can find our "Hello Java!" in this table as const #3 and const #18. I have no idea why it is split this way (perhaps a const for the reference and one for the char array?). I’ll leave that for the reader to explain to me. :)

Next notice the code below the constant pool. These are the byte code mnemonics of the program. Let’s focus on the main method where we see four things happen:

  1. Get the static field out from java.lang.System.
  2. Do something cryptic with our "Hello Java!" string.
  3. Invoke the method println on java.io.PrintStream with an argument of type java.lang.String.
  4. Finally, we return from the method.

Notice that the invokevirtual has neither System.out or "Hello Java!" specified. That is because in the previous two steps we put those values on a stack. The JVM is a stack-architecture machine. Every operation we have here (except return) operates on this stack. First we push the target object onto the stack, namely System.out. Then we load the argument string onto the stack from the run-time constant pool with ldc. Finally we invoke the appropriate method on the target object. This invocation pops those two items off the stack and performs the method.

Don’t confuse this stack with the call stack that you are familiar with. Each frame of the call stack has this stack I am referring to as its playground where it can push and pop values to perform all of the duties of the running program.

The main takeaways are (1) javap for viewing byte code, (2) the constant pool, and (3) the stack architecture. Stay tuned for the next installment where I will introduce some tooling to assist us in writing our own Hello World straight to JVM instructions.

Leave a reply below, or send me a tweet.

Tagged with: , ,
Posted in Software Development

Favoring expressions over statements

Two weeks ago today, I gave my CoffeeScript talk for the first time at a No Fluff Just Stuff tour event. Despite being received well both locally and at NFJS (sans one rather scathing review), I still feel like I’m not doing a good job explaining the difference between statements and expressions (on this point, I agree with the aforementioned negative review). In this blog post, I’ll sound-off my take on the difference in hopes that my followers can help me hone this into a solid and concise treatment. Bear in mind that my intended audience consists of programmers who have long been steeped in the tar pit of imperative programming. A lengthy excursion into formal lambda calculus is off the table, for instance, so I’m aiming for something colloquial. My goal is to nudge my audience’s thinking towards favoring expressions over statements, not to impress someone with big, college-education words.

What is a statement? In programming, this is an executable line of code. When you make a statement in a program, you are bidding the computer to do some action. In higher-level languages such as C, Java, JavaScript, etc., these typically translate down to several machine statements using assembly, bytecode, etc. Regardless of the compilation/translation process, these statements are intended to manipulate the state of the running program. Each statement changes a variable (hence a register and/or a value in memory), reads some input, or produces some output.

Expressions are a fundamental component of statements. Before a statement is executed, it must first determine the value which is to be involved. For instance, an assignment statement involves evaluating the expression on right-hand side of the operator before taking the action of assigning. Consider this trivial case of initializing a counter to zero: int i = 0. The 0 on the right hand side is a trivial expression which cannot be reduced further. We of course can have more interesting assignments such as int seconds = 60 * 60 * 24 or even better, double area = 3.14 * r * r. In these cases, the computer has a little work to do in order to reduce the expression down to a value, then perform the state change.

So what is an expression? It is a mathematical construct which can be reduced down to a simple value. Let’s not quibble about vocabulary here and focus on the last word: value. Expressions reduce to values. There is no notion of a program or state to manipulate. It can exist in a vacuum unlike a statement which requires this notion of program and state which can be manipulated.

Although CoffeeScript isn’t a purely expression-oriented language, it is more expression-oriented than it’s JavaScript counterpart. Let’s take a look at two if/else blocks of code which illustrate how we can replace statements with expressions. First, an if/else statement which sets min to the minimum value between a and b.

if a < b
  min = a
else
  min = b

Compare that to an if/else expression which sets the max.

max = if a > b then a else b

While both can be means to the same end, the latter shows us that if/else can just well be an expression, i.e. something which reduces to a value to which we assign the name max. Imperative languages know that we need this expressiveness. Ever heard of the ternary operator?

max = a > b ? a : b;

Unfortunately the ternary operator is a superfluous special case which shouldn’t be needed. Many of the existing constructs in programming languages could be expressions. Take for instance, the switch statement (as Richard Minerich points out here for C#/F#). CoffeeScript lets it be a statement.

lunch = switch day
  when "Mon","Tue","Thr" then "salad"
  when "Wed" then "wings"
  when "Fri" then "happy hour"
  else "sandwich"

If this were JavaScript, I’d have to insert the same lunch = text all in it. That is because if/else and switch statements originate from a paradigm founded in thinking in terms of a computer which must move things, execute commands, branch, etc. Expressions on the other hand are founded in mathematics. if/else/switch all translate well to mathematical functions, which are thoroughly studied and well-understood.

It turns out that expressions are generally easier to model logically and hence are more brain-friendly. This probably comes across as counter-intuitive to the masses who demand practical applications and scorn the rigors of mathematics. However, imperative programming is the result of years of dealing with the difficulties of the Von Neumann architecture by increasing abstractions. Expression-oriented programming (and hence functional programming) are the conclusion of decades of logical reasoning and deduction. Even for those who shy away from the abstract thinking involved in formal logic, it should be rather telling that these “practical” languages have a long history of striving towards abstraction, not away. The more I delve into programming with abstract expressions, the less I find myself in a mental battle to program. It flows much more naturally as it aligns with logical thought patterns.

Don’t buy it? Let’s also consider my previous statement about how expressions can exist in a vacuum. They have meaning without the context of the program. Statements on the other hand, do not. Sure, we can understand that a statement like println("Hello World!"); prints the obligatory first text to the console, but my point is that we include the notion of a program here. Expressions are simpler than statements because there are fewer components which are inherently involved. I would argue that simpler is always better. (See also, Simple Made Easy)

The more a language gives you expressions, the more you get to work with plain ol’ values, and the easier it is to reason about your code. While CoffeeScript offers mostly cute syntactic sugar, the strong gravitational pull towards more expressions is its greatest value proposition.

Leave a reply below, or send me a tweet.

Tagged with: , , , , , ,
Posted in Software Development

Video: 15-Minute Chat with lift-ng

This past Friday, I recorded myself practicing the short version of my presentation of lift-ng, and I thought I’d share with my blog followers. I used a Google+ live hangout to record it, and I now see that the first few moments are not recorded, which is why the video starts in the middle of a sentence. Nonetheless, the video looks/sounds quite good and it prepped me well to present it in Santiago. Enjoy!

Tagged with: , , ,
Posted in Software Development

Why a JVM on a VM?

This morning during my workout-procrastinating flip through my twitter feed, I came across Thoughts on Five Years of Emerging Languages. It is a fairly quick read which takes an interesting angle on the rather pedestrian set of language features implemented in Go as outlined in this article. It was certainly thought-provoking and has me reconsidering my disregard for Go. But that is a topic for another blog post.

Shortly after I retweeted that article, my mutual follower Gerbrand van Dieijen pointed out something that didn’t immediately jump out at me during my first reading:

That is an outstanding question. I’ve long believed in the value of hijacking the JVM despite the impedance mismatches languages like Scala and Clojure encounter. But for most of the past year all of the code I’ve shipped has been deployed in AWS on a VM. What’s the value in slapping a VM on top of that? All I need is something that runs on Linux that I can ship in Docker.

Recent developments in the Scala community certainly show the cost of running on the JVM. Typesafe’s plan for Scala 2.12 focuses on Java 8 interoperability. This of course has lead to the infamous forks of the Scala compiler. I’d wager a handsome bet this wouldn’t have happened if we weren’t paying the technical debt of our alien invasion of the Java virtual machine.

Neither Gerbrand or I are about to abandon the JVM, of course…

Although I’m beginning to understand what Brian McKenna meant by “mass delusion” of the value of the JVM, abandoning it would certainly be throwing the baby out with the bath water. One nice thing about the JVM in particular is getting to leverage all of those extant libraries. I’m not interested in a new ecosystem where everything has to be rebuilt like in Go. However, any well-established languages outside the JVM garner much more of my attention now. Haskell, anyone?

Tagged with: , , ,
Posted in Software Development

JavaScript is NOT a functional language

This humorous tweet garnered a plenty of attention early last week, and I confess I contributed to the virus with my own retweet. While I’m inclined to concur with the sentiment, I’m not completely comfortable doing so. To chose a side in the discussion implies not only that functional programming is a well-defined paradigm, but also that it is a Boolean property of languages. I’m not so sure we’ve all agreed what functional programming is, nor can we say that a language wholly is or isn’t functional.

My stance is the statement is neither true nor false, at least in the way it was meant to be interpreted. I certainly agree that JavaScript is more functional than, say, Java (especially prior to version 8), but it is also much less functional than the golden standard Haskell. Notice that I am making comparisons of the languages relative to one another, rather than partitioning the languages into opposing camps. I personally believe this is the best way to view a language’s place in functional programming as a position along continuum with imperative on one end and functional on the other.

On one end we have languages which we all agree aren’t functional, such as Small Talk or Pascal. The other end of the spectrum features the heralded functional language greats Lisp, Erlang, Haskell, ML, etc. What I find interesting is what lies in the space between those two. Some languages have properties which are certainly characteristic of the functional languages, but the language itself isn’t typically regarded as functional. Languages like Java are adopting features of functional languages making itself more functional. At what point does a language cross this mystical threshold?

Not only do I believe this should be regarded as a continuum, I believe there is a better question to ask when judging where the language lies: How much does this language help me write in a functional programming style? Let’s face it, Java will fight you to the death if you want to be functional, but the advent of lambdas and powerful libraries like Google’s Guava allow you to be quite a bit more functional than classic OOP Java.

What properties help us write in a functional style? A few that immediately come to my mind are (1) treating functions as first-class citizens, (2) declarative constructs, (3) bias towards immutable data, and (4) control over side-effects. This is by no means meant to be exhaustive, but properties like these ultimately combine to allow programs to achieve referential transparency, which is arguably the most important property of a functional program. This allows the developer(s) to reason about the program. The more a language favors these patterns as the idiomatic approaches to programming, the further towards functional programming end of the spectrum we should regard it.

Back to JavaScript, which is the title of this post. It does very well in the first property I feel we need, which is treating functions as first class citizens. However, it fails miserably on the next three. Many languages like Java at least give us a final key word, but everything is mutable in JavaScript. That’s not to say that you cannot write a program which does not mutate the data. It just means the language is not on your side to help you accomplish that goal. Hence, it makes sense to write an outstanding book like Michael Fogus did with Functional JavaScript. Yet the reader should be aware that it will take much discipline to retain the ultimate goal of referential transparency.

Another thing to notice is how some languages need libraries to allow the developer to write in a more functional style. I already mentioned Guava for Java. The Functional JavaScript book utilizes underscore.js. This shows me that sometimes a language needs a little help to be functional, but it can be done.

Perhaps also of interest is how much a language will allow you to NOT be functional. For instance, Scala is widely regarded as a functional language. Yet you can write some very filthy imperative code thanks to var declarations and easy-to-reach side-effecting calls such as println. Compare that to Haskell which requires you to rope off the entire function with caution tape like a crime scene to allow an I/O operation to occur. Hence, we could use this to say that Haskell helps developers write more functional code than Scala.

So I guess I agree that JavaScript isn’t a functional language, but only in the spirit that I don’t think it makes a lot of sense to declare languages as such, one way or the other. Rather we should make statements like “JavaScript helps me write in a functional style more than Java, but not as much as CoffeeScript which allows me write with more declarative expressions.” Even my most recent post has me regarding my ideal language as being functional. However, I think from now on I’m going to adopt the verbiage of stating how much I believe a language helps me write functional-style programs. I suspect this mentality will be more useful for discussing programming languages. It strikes me as being a little more objective therefore less likely to spark passionate-yet-pointless arguments about language design.

Leave a reply below, or send me a tweet.

Tagged with: , , , , ,
Posted in Software Development

If I were writing a language

I’ve never honestly put thought into what I would want in my own language until recently. For years I wore blinders while working contently in the world of Java. However my shift to Scala and full-stack development over the last two years has exposed me to a little of the world outside. I doubt I’ll soon find time to dabble with language design. But if I did, here are the features, concerns, and priorities I’d pursue.

Functional

My language would be functional, of course. I don’t know that I would attempt the OO hybrid that Scala has going on. Polymorphism indeed has its problems and I don’t want to deal with those. Clojure’s ad-hoc polymorphism is very intriguing, and I just don’t feel that data needs to be hidden. Functional is where the future is, and frankly it’s how I like to code.

Static typing

Yeah, duh. If you follow me at all, you know how I feel on this topic. I’m curious if this makes the language more difficult to design and implement. It seems obvious that it would given that it’s one more responsibility of the language. On the other hand, perhaps those guarantees buys some nice properties further down the line. Regardless, I’d like my resulting language to have the fantastic properties afforded by static typing. Seeing it done well in Scala and taken to the next level in Haskell makes me believe it’s well worth the cost.

Compilation targets

If I’m gonna write a language (and I probably won’t) I would shoot the moon. There is no silver bullet in software, but I wouldn’t let that stop me from trying to write the language to rule them all. In my web-oriented mindset, this implies compilation to multiple targets so it can be used anywhere. First, I’d target the JVM. It is a very impressive piece of engineering and would open up many platforms. In particular, it would be viable as a backend language and as a mobile language on Android. The second compilation target would of course be JavaScript. I’m still not a huge proponent of JS compilation, but the idea of compiling to a limited and optimized subset of JS is attractive. Although JavaScript has it’s good parts, it is absolutely riddled with filth. Frankly, there is no getting away from it in our browser world. I would also be keen on compiling to some native, perhaps LLVM would suffice. Ultimately, I just want it to compile to something which can run on iOS. Then all of the bases would be covered.

Web-readiness

In keeping with the previous section, my language would need to be web-ready to every extent possible. Compilation targets of the JVM, JavaScript, and LLVM are for web-readiness. The trickiest part will the the FFI, of course. I think I would prioritize great FFI with JavaScript/JSON. I know it runs against the static typing grain, but I really dig the simplicity of JavaScript objects. They’re all just a map. What I want is a way to be type safe, yet be able to utilize JavaScript objects with a native feel. I really don’t know how this would pan out. What I want to avoid is what we have in Scala, where we have case classes for our server-side data, then (assuming Lift) we have our JsonAST objects and the translations in between. What I imagine is something like all objects having a constructor which takes a JSON and instantiates the class. How do I reconcile that with static typing? No clue. Perhaps JSON-ready objects would be a special case of object. I’d have to play with it.

Also for FFI and interoperability, I would definitely make those calls safe. Scala’s interop with Java is super smooth and easy, but it leaves much to be desired by letting all of Java’s exceptions and nulls leak in. In my language, any calls to Java or JavaScript APIs would be wrapped in a monad like Lift’s Box which can be Empty (analogous to Scala’s None), Error, or Full[A] containing the result you hoped for (analogous to Scala’s Some).

Is Purescript close enough?

Perhaps so. I’ve been intrigued by Purescript lately, but have not committed any time to try it. It is strongly, statically typed and influenced by Haskell, so I know it’ll hit my first two items. It compiles to Javascript, and understand that the semantics are close to JavaScript. I’m optimistic that the FFI would be pretty good in that case. I’m not the first interested in a JVM compilation target either. I stay pretty busy in my free time as it is, and I’ve only made it worse by recently becoming a Lift committer and writing articles for NFJS Magazine. I’ve at least taken the first step and purchased PureScript by Example by Phil Freeman.

Those are a few quick things I would consider anyway. Most everything else about a language such as syntax and such isn’t that important to me. I just need these couple of things to build what I like building. What about you? What would you want in your own language? Or what languages do you think come close to what I already have in mind?

Leave a reply below, or send me a tweet.

Tagged with: , , , , , , , , ,
Posted in Software Development
Follow

Get every new post delivered to your Inbox.

Join 737 other followers