We must be the change you want to be

Grow

• combine with Prolog/minikanren and APL (a la Shen) https://alexalejandre.com/notes/unity-of-paradigms/
• OS features like DB, networking (a la plan9)
• sh-dsl and joy features
• dissect besides disasm
• J locales might be the best scoping and namespacing
  • in Q typing a name space in the REPL tells you what’s there
• Janet already has short funcs with predefined variables, but maybe there’s a more elegant approach - k defaults to 3 predefined inputs, Q lets you define inputs then do point free stuff
  • point free coe is an AST already, easily optimizable, however hooks and forms make you count verbs to the left and right
• K/Q have strong IPC, you make many processes constantly communicating together
• errors handling? https://lobste.rs/s/ce0ksl/second_great_error_model_convergence

jubilation about transducers overlooked that they are just a less generic form of ad-hoc polymorphism, invented to abstract over operations on collections

APL isn’t really one of these exhibitions of computational simplicity in the way of the languages you mention. It’s inventor, Kenneth Iverson, was more focused on the human side of thinking in and using the language. Forth, Lisp, et al are quite easy to implement, but they require considerable library layers on top to make them useful for expressing application-level logic, even if we just focus on the pure functions. APL, on the other hand, has a larger core set of primitives, but you’re then immediately able to concisely express high-level application logic. - xelxebar

Though succinct notation is a tool of thought, the operator set (having found its truest expression in BQN?) is key, raising the level of abstraction. I promose allowing longhand names and symbolic or ASCII shorthand. Array manipulations work just as well on tables, whether in K, Fennel or…

K forgoes rank fine in its target problem space, but some problems benefit. Which problem types benefit from rank? I suspect rank will be desirable - but perhaps through split primitives to avoid complexity in simple (far more common) cases.

Bakpakin does not approve of renaming things much, although the community often did think and discuss it.

; # break holy and sacrosanct tradition!

• weird naming:
  • https://github.com/janet-lang/janet/issues/626
  • https://github.com/janet-lang/janet/discussions/1251
  • any? is not a predicate but a functional or, also any
  • any, some, all
  • inconsistent theming: freeze, thaw vs immutable
  • sort, sorted, yet reverse, reverse! Bakpakin doesn’t want many !
    • more without a ! put, set, update, array/push, array/insert, array/remov
    • n.b. they return stuff, so they’re useful in pipelines etc.
  • chr confuses people who want it named byte
  • accumulate - could be named scan or folding-map
    • reduce, reduce2, accumulate2
  • keep-syntax and keep-syntax! - the ! isn’t mutable but rather coerces the types!

change by accretion, add new, don’t break old (this made me think of a rationalization library, aliasing functions with better names e.g. set!)

• next - i think this is effectively the same as asking if something is not empty
• invert - i think this can be used to make a table from an indexed value that is effectively a sort of set

I am reluctant to use these as-is in my own code to express the meanings above because i suspect i will rapidly forget their intent…thereby leaving my future self with puzzles i could have avoided

Such fixes are blocked to not break holy backwards compatability, which right, nevertheless sad. It is currently possible to just alias over (almost) everything and e.g. manipulate doc strings but that serves little purpose without fundamental improvements.

I thing the problem with adopting the ! suffix is that so many functions in the core mutate inputs that it would just become extra noise in the code. It works in Clojure because most functions don’t mutate inputs.

Anyways, what I will not do is append ! to all functions that mutate their input for the sake of bike shedding. - Bakpakin

Implementation

• bakpakin said the current memory model is somehow unfixable (without breaking), so fix it
• implement directly on linux system calls to have no dependencies, not even c? https://github.com/lone-lang/lone but then how to extend? But cffi could be necessary?
• intermediary representation / simplified assembly based on Common Lisp or Go’s, maybe? Wasm? Or what to do with the bytecode?
• Arthur Whitney stye c is lovely, but do it in zig. I don’t think it can be done in Rust (for memory and succinctness issues.) Potentially c would make sense, but…
  • macros climb the abstraction ladder. Reading you consume definitions from the top down
  • brevity leads to expressivity
  • ’thinking big’ in programming
• k has the best practices, breaking many “truths” industry holds

Every character carries a lot of meaning. A line of J could replace a page of just about anything else. In any other language, your brain ain’t working half the time; instantiating things, making iterators go, building brain dead switch statements, dealing with preposterous function call overhead or declarations, writing dumb helper patterns that are a tiny variation on something you have done 100 times before. With J you have to pay attention at all times. The line-noise look of the language makes you think better. I’m hoping it also makes you more productive; I figure a page of code a day is a decent amount of output; a page of J will do a lot of useful work. - Scott Locklin

Your CPU has a shockingly small amount of directly-attached memory (around 64kb). That’s been true since the 1970s, and is likely to be true for another fifty years. This used to be called RAM, but now we call anything directly addressable by the CPU as RAM, and we call this thing cache (or L1 or whatever) because that’s basically how most people use it. [1]

Now what your CPU does all day, is look at a memory address, and if that’s not in the cache, it sends a message to a memory controller and waits for that memory to show up (somewhat a simplification). Those instructions show up, now your CPU can execute them, and those instructions require more memory so again the CPU sends a message to the memory controller and waits for that memory to show up. All day long, alternating between these two wait states, we wait a bit for code, we wait a bit for data, and then we repeat.

Because K is small: Because the interpreter, the application source code, and all your state variables fit into that 64kb, you can see that memory waiting drops by half. - geocar

k’s fast (for an interpreter), because instead of for example a bytecode interpreter where each instruction operates on a single value, APL/j/k interpreters execute operators over whole arrays at once with optimized C code. - ehallewicz2

the set of operators has been designed to work with each other and cover the basic concrete needs (i.e what the CPU actually needs to do rather than how you would describe it in words). ,// is a complete implementation of “flatten”. |/0(0|+)\ is a complete implementation of an efficient the maximum-subarray-sum. In both cases, each character does one very well defined thing implemented in tight C (sometimes SIMD) code, and K interpreter just orchestrates how they interact. - beagle3

Lisp macros are awesome because they condense/abstact away repeated code. What’s the optimal function and macro set for productive coding? I suspect many array operators are in it. Much Lisp effort’s gone to finding the smallest primitive set - not the set leading to smaller solutions.

Is there something about the primitives that APL-derivitives provide that mean that more of your program can happen inside the primitives so the interpreter is doing less work?

Yes. Well, maybe not the primitives themselves, so much as how the language pushes you to use them.

J interpreter execution tends to be a few tight loops in built-in verb implementations, the loops are glued together with some interpretation-style higher-overhead dispatch code. Implicit looping incurs a once-per-array-op cost, but the cost of the computation done on each array element dominates the total cost. The interpreter is not especially clever, but the language encourages the programmer to write code that’s easy for the interpreter to handle.

Performance is a bit fragile though. If the operation you’re lifting over a whole array includes array-lifted operations internally, now you’re doing the heavy interpretive dispatch inside the loop. As a simple example, {.+{: means “add the first item to the last item.” If you apply it to a vector, it adds the first and last scalars. If you apply it to a matrix, it adds the first and last rows (and so on). Applying it to a very wide matrix is cheap because {. and {: just have to select the right rows, and then you have a simple loop over those rows’ elements. It’s about 166ms on my laptop with ten million row elements:

  wide =. i. 2 10000000
  6!:2 '({.+{:) wide'

0.166085ms

With the data oriented the other way, there’s no inherent reason computation should be any slower, but it is. ({.+{:)"1 means, “apply {.+{: to each individual vector.” So we’ve changed from selecting the top and bottom from each column to selecting the left and right from each row. The way the J interpreter is structured, we have to figure out the loop for {.+{:)"1, but the loop body isn’t just grabbing two numbers and adding them. The loop body jumps back to the interpreter to figure out how {.+{: should work on a 2-element vector. That takes about as long as figuring out how it should work on a 2-row matrix, but we do that for all ten million vectors.

  tall =. i. 10000000 2
  6!:2 '({.+{:)"1 tall'

2.99855ms

• kd0amg

Of simplicity in minimal OS like kOS:

Arthur writes pretty plain/linear-looking code, with little branching because duh, branching is usually slow. There are no modules or layers- the HID ISR writes the event data into a k object and calls k. The timer ISR is the same code. The frame buffer is memory mapped to the gpu but on the cpu-side has a k header in front. NVME hardware is memory mapped…

The main kos feature is what message sending is as expensive as negating a pointer and writing it to a memory address, and this is possible because all the code and global variables are in the higher half (negative pointers).

Why did people leave Janet?

Its a combination of language features out of the box and the frameworks and libraries at the time.

• How do I compress and resize images? In JS the winning library for image manipulation is sharp. But sharp isn’t available everywhere, like Cloudflare workers. Workers has their own image manipulation tool now too. 
• How do I hash something for deduplication? I solved that with janetls at the time. JS has its own language-built-in crypto set of functions. 
• How do I parse cookies? How do I authenticate cookie sessions or the like? 
• How do I do access control? (Firmly in the land of frameworks)
• How do I parse markdown and make my own html with it? How do I recognize my own special token to inject a component unique to my environment?
• And the usual “Hey this guy in IRC / Matrix made this library I like and suddenly they’re not around anymore”

Speaking of parsing, the tools Janet has are incredible. But nigh undebuggable. I found Parsec in Haskell to be much more enjoyable, honestly.

[1] Tangentially, is it true that modern caches (directly-attached memory) were called RAM in the past? From wiki:

Originally, PCs contained less than 1 mebibyte of RAM, which often had a response time of 1 CPU clock cycle, meaning that it required 0 wait states. Larger memory units are inherently slower than smaller ones of the same type, simply because it takes longer for signals to traverse a larger circuit. Constructing a memory unit of many gibibytes with a response time of one clock cycle is difficult or impossible. Modern CPUs often still have a mebibyte of 0 wait state cache memory, but it resides on the same chip as the CPU cores due to the bandwidth limitations of chip-to-chip communication.