Like a normal featureless cube, but sings comical songs.
387 stories
·
11 followers

John Rawls and the Original Position

1 Comment and 3 Shares
John Rawls: \
Read the whole story
emdeesee
12 days ago
reply
Wait...what?
Sherman, TX
Share this story
Delete

Roy Fielding’s Misappropriated REST Dissertation

1 Comment and 3 Shares

RESTful APIs are everywhere. This is funny, because how many people really know what “RESTful” is supposed to mean?

I think most of us can empathize with this Hacker News poster:

I’ve read several articles about REST, even a bit of the original paper. But I still have quite a vague idea about what it is. I’m beginning to think that nobody knows, that it’s simply a very poorly defined concept.

I had planned to write a blog post exploring how REST came to be such a dominant paradigm for communication across the internet. I started my research by reading Roy Fielding’s 2000 dissertation, which introduced REST to the world. After reading Fielding’s dissertation, I realized that the much more interesting story here is how Fielding’s ideas came to be so widely misunderstood.

Many more people know that Fielding’s dissertation is where REST came from than have read the dissertation (fair enough), so misconceptions about what the dissertation actually contains are pervasive.

The biggest of these misconceptions is that the dissertation directly addresses the problem of building APIs. I had always assumed, as I imagine many people do, that REST was intended from the get-go as an architectural model for web APIs built on top of HTTP. I thought perhaps that there had been some chaotic experimental period where people were building APIs on top of HTTP all wrong, and then Fielding came along and presented REST as the sane way to do things. But the timeline doesn’t make sense here: APIs for web services, in the sense that we know them today, weren’t a thing until a few years after Fielding published his dissertation.

Fielding’s dissertation (titled “Architectural Styles and the Design of Network-based Software Architectures”) is not about how to build APIs on top of HTTP but rather about HTTP itself. Fielding contributed to the HTTP/1.0 specification and co-authored the HTTP/1.1 specification, which was published in 1999. He was interested in the architectural lessons that could be drawn from the design of the HTTP protocol; his dissertation presents REST as a distillation of the architectural principles that guided the standardization process for HTTP/1.1. Fielding used these principles to make decisions about which proposals to incorporate into HTTP/1.1. For example, he rejected a proposal to batch requests using new MGET and MHEAD methods because he felt the proposal violated the constraints prescribed by REST, especially the constraint that messages in a REST system should be easy to proxy and cache.1 So HTTP/1.1 was instead designed around persistent connections over which multiple HTTP requests can be sent. (Fielding also felt that cookies are not RESTful because they add state to what should be a stateless system, but their usage was already entrenched.2) REST, for Fielding, was not a guide to building HTTP-based systems but a guide to extending HTTP.

This isn’t to say that Fielding doesn’t think REST could be used to build other systems. It’s just that he assumes these other systems will also be “distributed hypermedia systems.” This is another misconception people have about REST: that it is a general architecture you can use for any kind of networked application. But you could sum up the part of the dissertation where Fielding introduces REST as, essentially, “Listen, we just designed HTTP, so if you also find yourself designing a distributed hypermedia system you should use this cool architecture we worked out called REST to make things easier.” It’s not obvious why Fielding thinks anyone would ever attempt to build such a thing given that the web already exists; perhaps in 2000 it seemed like there was room for more than one distributed hypermedia system in the world. Anyway, Fielding makes clear that REST is intended as a solution for the scalability and consistency problems that arise when trying to connect hypermedia across the internet, not as an architectural model for distributed applications in general.

We remember Fielding’s dissertation now as the dissertation that introduced REST, but really the dissertation is about how much one-size-fits-all software architectures suck, and how you can better pick a software architecture appropriate for your needs. Only a single chapter of the dissertation is devoted to REST itself; much of the word count is spent on a taxonomy of alternative architectural styles3 that one could use for networked applications. Among these is the Pipe-and-Filter (PF) style, inspired by Unix pipes, along with various refinements of the Client-Server style (CS), such as Layered-Client-Server (LCS), Client-Cache-Stateless-Server (C$SS), and Layered-Client-Cache-Stateless-Server (LC$SS). The acronyms get unwieldy but Fielding’s point is that you can mix and match constraints imposed by existing styles to derive new styles. REST gets derived this way and could instead have been called—but for obvious reasons was not—Uniform-Layered-Code-on-Demand-Client-Cache-Stateless-Server (ULCODC$SS). Fielding establishes this taxonomy to emphasize that different constraints are appropriate for different applications and that this last group of constraints were the ones he felt worked best for HTTP.

This is the deep, deep irony of REST’s ubiquity today. REST gets blindly used for all sorts of networked applications now, but Fielding originally offered REST as an illustration of how to derive a software architecture tailored to an individual application’s particular needs.

I struggle to understand how this happened, because Fielding is so explicit about the pitfalls of not letting form follow function. He warns, almost at the very beginning of the dissertation, that “design-by-buzzword is a common occurrence” brought on by a failure to properly appreciate software architecture.4 He picks up this theme again several pages later:

Some architectural styles are often portrayed as “silver bullet” solutions for all forms of software. However, a good designer should select a style that matches the needs of a particular problem being solved.5

REST itself is an especially poor “silver bullet” solution, because, as Fielding later points out, it incorporates trade-offs that may not be appropriate unless you are building a distributed hypermedia application:

REST is designed to be efficient for large-grain hypermedia data transfer, optimizing for the common case of the Web, but resulting in an interface that is not optimal for other forms of architectural interaction.6

Fielding came up with REST because the web posed a thorny problem of “anarchic scalability,” by which Fielding means the need to connect documents in a performant way across organizational and national boundaries. The constraints that REST imposes were carefully chosen to solve this anarchic scalability problem. Web service APIs that are public-facing have to deal with a similar problem, so one can see why REST is relevant there. Yet today it would not be at all surprising to find that an engineering team has built a backend using REST even though the backend only talks to clients that the engineering team has full control over. We have all become the architect in this Monty Python sketch, who designs an apartment building in the style of a slaughterhouse because slaughterhouses are the only thing he has experience building. (Fielding uses a line from this sketch as an epigraph for his dissertation: “Excuse me… did you say ‘knives’?”)

So, given that Fielding’s dissertation was all about avoiding silver bullet software architectures, how did REST become a de facto standard for web services of every kind?

My theory is that, in the mid-2000s, the people who were sick of SOAP and wanted to do something else needed their own four-letter acronym.

I’m only half-joking here. SOAP, or the Simple Object Access Protocol, is a verbose and complicated protocol that you cannot use without first understanding a bunch of interrelated XML specifications. Early web services offered APIs based on SOAP, but, as more and more APIs started being offered in the mid-2000s, software developers burned by SOAP’s complexity migrated away en masse.

Among this crowd, SOAP inspired contempt. Ruby-on-Rails dropped SOAP support in 2007, leading to this emblematic comment from Rails creator David Heinemeier Hansson: “We feel that SOAP is overly complicated. It’s been taken over by the enterprise people, and when that happens, usually nothing good comes of it.”7 The “enterprise people” wanted everything to be formally specified, but the get-shit-done crowd saw that as a waste of time.

If the get-shit-done crowd wasn’t going to use SOAP, they still needed some standard way of doing things. Since everyone was using HTTP, and since everyone would keep using HTTP at least as a transport layer because of all the proxying and caching support, the simplest possible thing to do was just rely on HTTP’s existing semantics. So that’s what they did. They could have called their approach Fuck It, Overload HTTP (FIOH), and that would have been an accurate name, as anyone who has ever tried to decide what HTTP status code to return for a business logic error can attest. But that would have seemed recklessly blasé next to all the formal specification work that went into SOAP.

Luckily, there was this dissertation out there, written by a co-author of the HTTP/1.1 specification, that had something vaguely to do with extending HTTP and could offer FIOH a veneer of academic respectability. So REST was appropriated to give cover for what was really just FIOH.

I’m not saying that this is exactly how things happened, or that there was an actual conspiracy among irreverent startup types to misappropriate REST, but this story helps me understand how REST became a model for web service APIs when Fielding’s dissertation isn’t about web service APIs at all. Adopting REST’s constraints makes some sense, especially for public-facing APIs that do cross organizational boundaries and thus benefit from REST’s “uniform interface.” That link must have been the kernel of why REST first got mentioned in connection with building APIs on the web. But imagining a separate approach called “FIOH,” that borrowed the “REST” name partly just for marketing reasons, helps me account for the many disparities between what today we know as RESTful APIs and the REST architectural style that Fielding originally described.

REST purists often complain, for example, that so-called REST APIs aren’t actually REST APIs because they do not use Hypermedia as The Engine of Application State (HATEOAS). Fielding himself has made this criticism. According to him, a real REST API is supposed to allow you to navigate all its endpoints from a base endpoint by following links. If you think that people are actually out there trying to build REST APIs, then this is a glaring omission—HATEOAS really is fundamental to Fielding’s original conception of REST, especially considering that the “state transfer” in “Representational State Transfer” refers to navigating a state machine using hyperlinks between resources (and not, as many people seem to believe, to transferring resource state over the wire).8 But if you imagine that everyone is just building FIOH APIs and advertising them, with a nudge and a wink, as REST APIs, or slightly more honestly as “RESTful” APIs, then of course HATEOAS is unimportant.

Similarly, you might be surprised to know that there is nothing in Fielding’s dissertation about which HTTP verb should map to which CRUD action, even though software developers like to argue endlessly about whether using PUT or PATCH to update a resource is more RESTful. Having a standard mapping of HTTP verbs to CRUD actions is a useful thing, but this standard mapping is part of FIOH and not part of REST.

This is why, rather than saying that nobody understands REST, we should just think of the term “REST” as having been misappropriated. The modern notion of a REST API has historical links to Fielding’s REST architecture, but really the two things are separate. The historical link is good to keep in mind as a guide for when to build a RESTful API. Does your API cross organizational and national boundaries the same way that HTTP needs to? Then building a RESTful API with a predictable, uniform interface might be the right approach. If not, it’s good to remember that Fielding favored having form follow function. Maybe something like GraphQL or even just JSON-RPC would be a better fit for what you are trying to accomplish.

If you enjoyed this post, more like it come out every four weeks! Follow @TwoBitHistory on Twitter or subscribe to the RSS feed to make sure you know when a new post is out.

Previously on TwoBitHistory…

  1. Roy Fielding. “Architectural Styles and the Design of Network-based Software Architectures,” 128. 2000. University of California, Irvine, PhD Dissertation, accessed June 28, 2020, https://www.ics.uci.edu/~fielding/pubs/dissertation/fielding_dissertation_2up.pdf

  2. Fielding, 130. 

  3. Fielding distinguishes between software architectures and software architecture “styles.” REST is an architectural style that has an instantiation in the architecture of HTTP. 

  4. Fielding, 2. 

  5. Fielding, 15. 

  6. Fielding, 82. 

  7. Paul Krill. “Ruby on Rails 2.0 released for Web Apps,” InfoWorld. Dec 7, 2007, accessed June 28, 2020, https://www.infoworld.com/article/2648925/ruby-on-rails-2-0-released-for-web-apps.html 

  8. Fielding, 109. 

Read the whole story
emdeesee
43 days ago
reply
I mean, yeah, alphabet soup, but kind of interesting alphabet soup.
Sherman, TX
Share this story
Delete

Authorial Quirk

1 Comment
Read the whole story
emdeesee
53 days ago
reply
Did the plot thicken? It feels like the plot just thickened.
Sherman, TX
Share this story
Delete

queeranarchism: pblomgr1: These are the solutions we need to...

9 Shares




















queeranarchism:

pblomgr1:

These are the solutions we need to policing right now.
Remember: the problem cannot be solved by technocratic solutions (i.e. body cams, further trainings, etc.)
The problem is policing itself.

This is the sort of shit I am talking about when I say we need to only talk about getting rid of police but also about what sort of actual safety could replace it.

And no, it is not enough for this to exist ‘next to the police’. The harm is in the fact that circumstances of personal and interpersonal harm are viewed through the lens of law and punishment. 

The moment we take the concept of laws & punishment completely off the table and start thinking in needs and how to provide them, we become capable of seeing what is needed to achieve actual safety.

Read the whole story
emdeesee
57 days ago
reply
Sherman, TX
popular
57 days ago
reply
jepler
57 days ago
reply
Earth, Sol system, Western spiral arm
Share this story
Delete

Alexander Artemenko: cl-collider

1 Comment

This library is an interface to a very interesting algorithmic audio synthesizer - SuperCollider. SuperCollider itself has a builtin programming language, but cl-collider makes it possible to write music in lisp interactively.

Here is how a simple program in sclang looks like:

// 60Hz Gabber Rave 1995
Server.default.boot;

(
SynthDef(\gabberkick, {
    var snd, freq, high, lfo;
    freq = \freq.kr(440) * (Env.perc(0.001, 0.08, curve: -1).ar * 48 * \bend.kr(1)).midiratio;
    snd = Saw.ar(freq);
    snd = (snd * 100).tanh + ((snd.sign - snd) * -8.dbamp);
    high = HPF.ar(snd, 300);
    lfo = SinOsc.ar(8, [0, 0.5pi]).range(0, 0.01);
    high = high.dup(2) + (DelayC.ar(high, 0.01, lfo) * -2.dbamp);
    snd = LPF.ar(snd, 100).dup(2) + high;
    snd = RLPF.ar(snd, 7000, 2);
    snd = BPeakEQ.ar(snd, \ffreq.kr(3000) * XLine.kr(1, 0.8, 0.3), 0.5, 15);
    snd = snd * Env.asr(0.001, 1, 0.05).ar(2, \gate.kr(1));
    Out.ar(\out.kr(0), snd * \amp.kr(0.1));
}).add;

SynthDef(\hoover, {
    var snd, freq, bw, delay, decay;
    freq = \freq.kr(440);
    freq = freq * Env([-5, 6, 0], [0.1, 1.7], [\lin, -4]).kr.midiratio;
    bw = 1.035;
    snd = { DelayN.ar(Saw.ar(freq * ExpRand(bw, 1 / bw)) + Saw.ar(freq * 0.5 * ExpRand(bw, 1 / bw)), 0.01, Rand(0, 0.01)) }.dup(20);
    snd = (Splay.ar(snd) * 3).atan;
    snd = snd * Env.asr(0.01, 1.0, 1.0).kr(0, \gate.kr(1));
    snd = FreeVerb2.ar(snd[0], snd[1], 0.3, 0.9);
    snd = snd * Env.asr(0, 1.0, 4, 6).kr(2, \gate.kr(1));
    Out.ar(\out.kr(0), snd * \amp.kr(0.1));
}).add;
)

(
var durations;
durations = [1, 1, 1, 1, 3/4, 1/4, 1/2, 3/4, 1/4, 1/2];
Ppar([
    Pbind(*[
        instrument: \gabberkick,
        amp: -23.dbamp,
        freq: 60,
        legato: 0.8,
        ffreq: Pseq((0..(durations.size * 4 - 1)).normalize, inf).linexp(0, 1, 100, 4000),
        dur: Pseq(durations, inf),
        bend: Pfuncn({ |x| if(x < (1/2), 0.4, 1) }, inf) <> Pkey(\dur),
    ]),
    Pbind(*[
        instrument: \hoover,
        amp: -20.dbamp,
        midinote: 74,
        dur: durations.sum * 2,
        sustain: 7,
    ])
]).play(TempoClock(210 / 60));
)

I wasn't able to translate it into the Lisp form, because cl-collider does not have documentation and diving into it will require too much time. However, there is a working code example from its README, which you can try in the REPL:

POFTHEDAY> (named-readtables:in-readtable :sc)
POFTHEDAY> (setf sc:*s* (sc:make-external-server
                         "localhost" :port 48800))
POFTHEDAY> (use-package :sc)

POFTHEDAY> (defsynth saw-synth ((note 60) (dur 4.0))
             (let* ((env (env-gen.kr (env [0 .2 0]
                                          [(* dur .2) (* dur .8)])
                                     :act :free))
                    (freq (midicps note))
                    (sig (lpf.ar (saw.ar freq env)
                                 (* freq 2))))
               (out.ar 0 [sig sig])))

POFTHEDAY> (defun make-melody (time n &optional (offset 0))
             (when (> n 0)
               (at time (synth 'saw-synth
                               :note (+ offset (alexandria:random-elt
                                                '(62 65 69 72)))))
               (let ((next-time (+ time (alexandria:random-elt
                                         '(0 1 2 1.5)))))
                 (callback next-time #'make-melody
                           next-time (- n 1) offset))))

POFTHEDAY> (make-melody (quant 4) 16)
POFTHEDAY> (make-melody (+ 4 (quant 4)) 16 12)

;; This will stop the music
POFTHEDAY> (sc:stop)

Here is a demo, showing how does live coding works with cl-collider:

https://www.youtube.com/watch?v=pZyuHjztARY

To try cl-collider, you'll need to install a SuperCollider. On OSX it is as simple as doing:

brew cask install supercollider
Read the whole story
emdeesee
77 days ago
reply
I guess it's time to try this out again. Last time, I failed to get SuperCollider working. I was much more interested in the library and probably didn't pay enough attention to SC's docs.
Sherman, TX
Share this story
Delete

Michał Herda: Parentheses and indentation

1 Comment

Claim: You know you've got used to reading Lisp when you no longer care about the parentheses and instead read Lisp by indentation. And this is how it is supposed to be read.

(Warning: this post has a slight rant tone to it.)


Let us consider three versions of read-file-into-string, a Common Lisp utility function adapted from the Alexandria source code. The questions are: How are they different? How do they work? What do they say about the code that is executed?

            ;;; Version A

(defun read-file-into-string (pathname &key (buffer-size 4096) external-format)
  (flet ((read-stream-content-into-string (stream)
           (check-type buffer-size (integer 1))
           (let ((*print-pretty* nil)
                 (buffer (make-array buffer-size :element-type 'character)))
             (with-output-to-string (datum)
               (loop :for bytes-read := (read-sequence buffer stream)
                     :do (write-sequence buffer datum :start 0 :end bytes-read)
                     :while (= bytes-read buffer-size))))))
    (with-open-file (stream pathname :direction :input
                                     :external-format external-format)
      (read-stream-content-into-string stream :buffer-size buffer-size))))

          
            ;;; Version B

(defun read-file-into-string ((pathname &key (buffer-size 4096) external-format)))
  (flet (read-stream-content-into-string (stream)
          (check-type buffer-size (integer 1)
          (let ((*print-pretty* nil))
                (buffer (make-array buffer-size :element-type 'character))
            (with-output-to-string (datum)))
              (loop :for bytes-read := (read-sequence buffer stream)
                    :do (write-sequence buffer datum :start 0 :end bytes-read))
                    :while (= bytes-read buffer-size)))
    (with-open-file ((stream pathname :direction :input
                                      :external-format external-format)))
      (read-stream-content-into-string stream :buffer-size buffer-size)))))))

          
            ;;; Version C

(defun read-file-into-string (pathname &key (buffer-size 4096) external-format)
  (flet ((read-stream-content-into-string (stream)
           (check-type buffer-size (integer 1))
    (let ((*print-pretty* nil)
      (buffer (make-array buffer-size :element-type 'character)))
      (with-output-to-string (datum)
        (loop :for bytes-read := (read-sequence buffer stream)
              :do (write-sequence buffer datum :start 0 :end bytes-read)
              :while (= bytes-read buffer-size))))))
        (with-open-file (stream pathname :direction :input
                                         :external-format external-format)
          (read-stream-content-into-string stream :buffer-size buffer-size))))

          

You are free to check these in a Common Lisp REPL in case of doubts.


The answer is that A and B tell the same story to the programmer, even though B won't compile. Many starting and ending parentheses in version B have been removed, duplicated, or displaced, which makes that code incomprehensible to the Lisp compiler.

C, however, does compile and work just like A does, and the Lisp compiler will not see any difference between forms from A and C. This is because C is a copy of A with broken indentation. The only thing that differs is the whitespace at the begining of each line.

To a Lisp programmer, version C is much more dangerous than B: while trying to evaluate the code from version B provides immediate feedback (it won't compile, it's broken code!), version C will instead silently work in a way that is not expected.


The intent conveyed by version A is that most of the space is taken by a local function, which is why most of the middle is indented more heavily than the bottom lines that form the actual body of read-file-into-string. Version C instead assumes that the only thing done by the local function is a check-type assertion - it is the only form indented in a way that denotes the body of a local function. The rest of function body implies that we first call some function named buffer on a freshly created array. Then, we open a with-output-to-string context, and perform everything else - which are the loop iteration and the subsequent with-open-file form - inside that context.

Such indentation is actively hostile towards the programmer, as I have intentionally created it to be misleading; it is therefore unlikely to find it in Lisp code that is actively used. Still, it is a proof of concept that it is possible to mislead a Lisp programmer, either by someone who either actively tries to do it or by one who is novice enough to not know better - and therefore, indirectly, a proof that indentation pays a crucial role in understanding and parsing Lisp code by humans.


To make another proof, we can take this example in another direction, a very extreme one this time. We will take the code from version A and remove all the parentheses from it (except where they are required to parse the context), leaving only indenation in place.

            ;;; Version D

defun read-file-into-string pathname &key (buffer-size 4096) external-format
  flet read-stream-content-into-string stream
         check-type buffer-size integer 1
         let *print-pretty* nil
             buffer make-array buffer-size :element-type 'character
           with-output-to-string datum
             loop :for bytes-read := read-sequence buffer stream
                  :do write-sequence buffer datum :start 0 :end bytes-read
                  :while = bytes-read buffer-size
    with-open-file stream pathname :direction :input
                                   :external-format external-format
      read-stream-content-into-string stream :buffer-size buffer-size

          

Suddenly, we get something strangely pythonesque. Code scopes are no longer defined by parentheses and instead they are defined purely by whitespace. Lisp programmers might also be put off by the sudden lack of parentheses.

And yet, this example - at least to me - reads very similar to the Lisp code from variant A. Again, this is because the indentation for both pieces of code is identical: it is clear where a given block or subblock goes deeper, where it continues at the same depth, and where it ends, and this is the information that a Lisp programmer uses when parsing code meant for human consumption.


There's a valid point that needs to be taken into account here - that one needs to be somewhat proficient Lisp semantics in order to know the argument counts for each form that is executed. In the above example, one needs to know that make-array takes one mandatory argument and may then take a number of keyword arguments, that write-sequence takes two mandatory arguments and keyword args, that check-type takes a variable and a type, and so on. Such knowledge comes only from using the language in depth, but, thankfully, it is knowledge that relates to the right side of each line of such program, rather than to the left part. And the left is where the indentation is.


When writing Lisp, two tasks are not meant to be done by humans: managing parentheses and indenting code. The programmer's focus should be on what happens between the parentheses and whose meaning and order is provided by indentation.

  • When I write Lisp, I do not pay any attention about the indentation; emacs automatically indents my code for me as I write it thanks to electric-indent and aggressive-indent.
  • When I write Lisp, I do not need to pay any attention to closing or counting parentheses: emacs automatically inserts them in pairs and prevents me from deleting a lone paren thanks to smartparens, and I have a visual color cue that informs me about the depth of a given paren thanks to rainbow-delimiters.
  • When I write Lisp, I do not need to pay much attention to where exactly I insert a new subform: if I misplace a form within my Lisp expression, emacs will automatically indent it for me, and I will notice that it is not indented at the level where I expected it to be. I can then fix its position, again, thanks to smartparens.

This is also why I consider writing Lisp code to be almost impossible without support from an editor that performs these tasks for the programmer. Even more, I consider sharing Lisp code to be impossible if that code is not formatted correctly, because then this code will not be indented correctly and will therefore either require other Lisp programmers who read it to put extra cognitive load to try and understand it, or, worse - it will give them outright wrong and misleading information about what a given piece of Lisp code does.


Oh, and while we're at it, Lisp also solves the tabs-versus-spaces debate. It is impossible to indent Lisp with tabs only, unless either tabs are exactly one-space wide or one chooses the worst possible option of both at the same time in the same lines.

So, spaces it is.

Read the whole story
emdeesee
82 days ago
reply
Though it's my favorite language, I'm not sure it's a feature that it's "almost impossible [to write] without [being a cyborg]."

OTOH I think I do a passable job matching parens (even writing longhand!), even while those same parens melt into the background.
Sherman, TX
Share this story
Delete
Next Page of Stories