Transcript
Transcript prepared by
Bob Therriault
'Pepe' Quintana
00:00-00:06
This is the perfect language for a lazy person. You can accomplish so much with so little effort.
Music
00:06-00:06
Conor Hoekstra
00:16-00:37
Welcome to episode 115 of ArrayCast. I'm your host, Conor, and today with us we have a special guest who I am very excited to talk to. He's been on our list for a long time and we finally have him here. But before we get to introducing him, we have our brief introductions and a single announcement. We'll start with Bob, then go to Stephen, then go to Marshall and finish with Adám.
Bob Therriault
00:37-00:45
I'm Bob Therriault, and today I am especially honored to have this guest because he's been a hero of mine for a while. I'm a J enthusiast.
Stephen Taylor
00:46-00:53
I'm Stephen Taylor and I'm very briefly an APL enthusiast and even more briefly a q enthusiast.
Marshall Lochbaum
00:54-01:03
I'm Marshall Lochbaum. I started out with J, including a brief stint working at Best LLC and I've been through some other languages and now I do BQN.
Adám Brudzewsky
01:03-01:09
I'm Adám Brudzewsky. I'm mainly doing APL, but J has always intrigued me since I was little.
Conor Hoekstra
01:10-01:27
and looking forward to hearing this. And as mentioned before, my name is Conor, host of ArrayCast, massive fan of all the Array languages, and very excited for today's conversation. Before I introduce our guest today, I believe we have one short announcement from Adám, I believe.
Adám Brudzewsky
01:27-01:56
Yeah, so...By the time this episode comes out, it's the absolute last moment to catch the DYNA physical meetup in New York City, which is on Monday the 29th and Tuesday the 30th of September. [01] And there's now a video on the DYNA website, DYNA.dyalog.com, where our CTO, Morten, introduces some of the subjects that will be discussed and presented. It's DYNA.
Conor Hoekstra
01:56-03:28
Awesome. Links will be in the show notes. If you're able to attend that, the details will be in the link. And with that single announcement out of the way, it is my extreme pleasure to introduce today our guest. He goes by Pepe. His full name is Jose Mario Quintana. And I've known about Pepe. I've never actually spoken to you one-on-one, but I've known about you from Bob since... Probably I got into array languages and I fell in love with tacit programming. Your name has come up time and time again as this mythical tacit J expert. And I'm super excited for this conversation. In preparation, Bob sent a couple links to us. I did not – I think maybe Bob had mentioned at one point that you had a – modified, executable. I did not know, though, that it was available online. This is Jx. And I didn't go through the full history of the discussion because there was a long email thread. I read a small portion that said explicit versus tacit that talked about this, but I am super excited today to talk to you Pepe, to talk about tacit programming to talk about Jx and these extensions that you have added to J and other things that I probably don't even know about. I'm going to stpp talking. I am gong to throw it over to you. Maybe for the listeners you could introduce yourself, if you want take us back to when you got into programming or into array languages and take us forward to how you ended up getting into J and deciding that there were some things missing in J such that you wanted to modify it.
'Pepe' Quintana
03:28-18:35
Thank you for having me. I follow the podcast. It's an honor to be here. I should mention that the Jx version that is online is a very old version. And we have modified, extended, but that is not being put available online. So I will start a little bit of my journey. It starts when I was in high school. I like math a lot. I was good at it. And the teacher, very good teacher, said, "Good mathematician has to be lazy by nature." I certainly qualified, but I also like money a little bit too much. So I thought, as a mathematician, I won't be able to make all the money that I want to make. And somebody, a friend of my brother, mentioned one day that, oh, there is this bachelor degree called actuary. And they use a lot of math and work for insurance companies and they pay a lot of money. but only a small fraction graduate. So I said, oh, that's actually an incentive. So I decided right there to study, to be an actuary. So I went to university, and then, of course, they teach Fortran and COBOL for technical programming, business programming, and those were run on the Burroughs 6500. And the main language for that machine was algol. And I used that language to write some programs for my thesis as an actuary. I was in a hurry, so I finished very quickly in three years, but I still have one class left. And together with another three actuaries and one that was studying physics, we said, let's try to see if we can make money in the racetrack betting on horses. So we tried to put a model. I learned a lot. And one thing I learned is to write a program to invert matrices. So I went after graduating, I started to work for a small brokerage house. I said, maybe better betting in the stock market. At that time, another working person, a colleague that was also from the university, said, look, there is this personal computer that came out. And actually, you can hook it to the TV and a keyboard and do a lot of things. That was an Apple 1. So we chip in 50-50 and we got it. Later I sold my part had I known that that Apple 1 computer at a museum would pay $900,000 for that thing. And then I discovered Bayesian statistics and making decisions under uncertainty and decided I need to learn more about this. So I got into a master's in statistics and operations research. My thesis was having new algorithms for a problem that is called a linear complementarity problem. And that is an interesting problem and very general. And one of the applications is quadratic programming. As a consequence, linear regressions with constraints. For that, I bought Apple II now, and I learned Pascal, and I used Pascal to develop those algorithms. During that time, I also do a consultancy, one for epidemiologists and another interesting one for a couple of economists that were implementing, they wanted to implement an input-output model for the Mexican economy. So that inversion of metrics is an algorithm that I had learned and became handy. But I realized I needed more and decided to go for a PhD in statistics, forecasting. And the best place at the time was in England, University of Warwick. So I went there. And right there I decided to get another personal computer, cheap one, it was called Sinclair-QL. Anyway, it had something called SuperBasic, it had re-emptive multitasking. And I started coding my research, I wanted to do multivariate Bayesian forecasting models, that's actually the name of my thesis. But while I was there, near the end, suddenly a bunch of brand new IBM PC's show up. What is this? Well, this is a gift from IBM. But there is a catch. They come loaded with this programming language called APL. So I look at it and I didn't understand this thing. All my background didn't help me and I ask some colleague, does it make sense to you? He said, well, look, some things can be done like this. This is where you can produce a histogram. And then I look at a bunch of symbols and all that and say, "Oh, maybe I should learn these symbols." And then I start to look more and I realized you can do matrix multiplication just like this, with a few symbols and you can solve linear equations and geometric symbols. I could have done in one line all that input-output model that I was working for those economies. This is the perfect language for a lazy person. You can accomplish so much with so little effort. So I was sold at that point. I went back to the States and I got a job at AT&T. They were using Unix and something called S. S, of course, is the precursor of S plus, which was the model for R, the open source version. And my boss decided to pursue art. That was her passion. I didn't care because to me it was just a way to get closer to New York and get a job in finance. My idea was to forecast the financial markets. So I interviewed with the investment arm of Chase, and they were setting up a currency program taking advantage of the convergence to the euro at that time. And there was an opportunity to make a good amount of money, they thought, but they needed to control risk. And one of my examples in my thesis was currency, so they hired me. Unfortunately, my boss was a SAS system fanatic. So he insisted that I use what it was called interactive matrix language, IML, part of SAS. I don't know if that qualifies as an array language, I don't know, but I didn't like it. [02] We used another system called GAMS, generalized algebraic modeling system. Anyway, that was for solving mathematical programming, that is optimization. My boss got another job. His boss became my boss and I convinced him to get APL. I said, we can do a lot of things faster using APL. I researched and we go with Dyalog APL. And that system was incredible. Well, first we did the research simulations, it looked great on paper. So the head of the unit went to the Middle East a few times and raised half a billion dollars. That thing was a money printing machine. It was really, really doing well. And then there was an acquisition. Chase decided to sell, well, not exactly the whole thing, to another bank. And everybody has a choice to stay there or move to the Chase Manhattan Bank. But they call me and say, you don't have an option. You are coming to the Chase Manhattan Bank because we're selling everything. But this program, we will implement them there. And that's it. So I follow there. And we extended to include bonds and stocks. And more factors now because that opportunity was fading away. It was just during the conversion to the single currency. One day they come and they call it a pizza box, but it was a sun workstation. And it came with Unix. And they were not going to pay for the APL. So I did research and I found J, an alternative or similar to APL. But that time I knew more about the program and I was involved in trading and other things. So eventually I wanted to be portfolio manager. And my boss was very good with me. So I told him, if I leave, this one will be in trouble. I am the only one that knows APL. So I told him, you know, I need some help. I need to hire somebody to help me with APL. And he looked at me like, are you crazy? He didn't say that you have the best job security of anybody here. But okay, if that's what you want, go ahead. And I heard somebody that you might be familiar with the name of the person is Devon McCormick. So, he worked for me for a while, and at that time I don't think he knew much about J. But we started talking and I told him at some point, "I suspect that everything can be coded tacit." And I was interested with that because especially compared with Dyalog APL, the explicit way in programming in J was really awkward. I didn't like it at all. So he said, "Well, you can try to code the Ackermann function. That's a tough one. See if you can do it." And then I look at it and it was a recursive definition. I said, "Well, that's a simple one. That's not a problem." Maybe a non-recursive version, that would be a challenge. Later I realized that you can use the Y combinator to implement it. I did that. And I even put a version later on in the Rosetta code of that. But that was a good thing, because I thought, "Well, if I manage to implement this thing directly, somebody else will come up with a harder problem. It's never ending. But in that page, they were saying, oh, Ackerman function is Turing computable. And then I say, oh, if I could implement the Turing computer tacitly, that means it's complete. And I can implement, in principle, everything. So in my commuting back from New York, I was able to implement that Turing computer emulator. And that is in two places, two versions. One again in the Rosetta code, you can find the Turing machine there, tacit version. Also in a conference, I talk about that in 2012, And I think if you use the right browser, you can still load my presentation. Not only I was able to implement, but then I realized I can know now how to implement everything. And then I have had the idea to now I can do all the system there. Dyalog was great, but I have a problem. Sometimes the workspace was corrupted. You couldn't see anything inside that. And it was frustrating. Also new versions, I didn't have time to catch up. Sometimes I couldn't even print. At some point I decided to port everything to J, tacit J, everything, not only tacit but fixed, so no names, everything tacit. So I coded this linear complementarity problem solution. And that's all the quadratic programming in general. And now everything was, the core was in tacit, all tacit. We were running for more than a decade until I decided to start slowing down. We moved to Florida and we dropped all the clients. And eventually we only have one client now and it's ourselves. We use it to manage our own money. So what is the idea in general? You can do a lot with tacit, you can manage some arguments, and you can do three, you can juggle it, but after that it's really hard. So the idea is to use functions or verbs to do that. And in traditional programming, you have your variables, you update it, and then you update more variables and you keep it going like that. Sometimes you have loops in those updating, but that's a general framework. And in J, when you are doing that tacit, you can emulate when it's necessary. When you have a lot of things, not just two, three, four, but several, you have to go that way. And the way to do it is with nested or boxed arrays. And the contents of that are like variables, the equivalent. And then the others, you can have functions to update those. And you update the whole state that contains all the equivalent of all the variables that you have to do. There are two ways, but the most practical way to do that is to use the amend primitive, so you can update parts of that, and you can use J to define the functions that you want to use. But in any case, this not only shows how it works, but a general way to do anything that you really have to go deeper than one line of tacit code.
Bob Therriault
18:35-18:42
And was the purpose of doing this to show that J tacit is Turing complete? Is that why you were pursuing this?
'Pepe' Quintana
18:42-19:33
That will show me that it's Turing complete. But at the same time, by seeing how is it done, it tells me how to do any other. I haven't encountered anything that cannot be done this way because that, in the worst case, replicate the usual way that you are doing in J or in any other programming language, updating variables one at a time. So the way to do it is with functions and updating the state that is a boxed array with the contents that emulate variables. So if it has to go that way, you can always do it like that. Again, you can emulate local variables by expanding and appending the extra boxes that you need to do it.
Bob Therriault
19:33-19:39
In terms of efficiency or speed, it... It probably isn't meant to be.
'Pepe' Quintana
19:40-20:49
Oh, no. J is probably one of the worst to implement a Turing machine because it's very little pieces. But if you are using in general J, the pieces, if you are doing it the right way, holds a lot of data. And so the arrays are big, at least in the cases where J can do things efficiently. But it doesn't matter what the size is. The framework is the same. And people say, "Oh, you should only use tacit for one-liners, but for other things you have to do explicit." Well, with this way you don't have to. Everything can be done tacitly. And again, if you hold a lot of data, then J is sufficient. To run in this Turing machine, oh, yeah, it's not the fastest, especially amending. Amending takes a long. If you do it in place, it will be better, but it's not easy to do amending in place. Actually, that's one of the extensions we did at some times, to do amending in place.
Bob Therriault
20:49-21:02
And I guess with Jx, that's when you split off, from the regular J, and I think it was, was it at J806 or is it J805 is the one that you have, the current?
'Pepe' Quintana
21:03-21:58
Yeah, you can, J806 still is good for us. [03] There is another reason for the split at that time. tacit programming, as you know, is controversial. Any verb can be done tacitly. To program tacitly, adverbs in conjunction, that was more tricky. I learned J a long time ago, so they were the original. They are called modifiers now, but... But even there, there were things that were hard to do. I couldn't figure it out. At the beginning, later, we found a way. With a lot of work that also, Dan Bron did a lot of things. I don't know if he's involved in J so much anymore. He's not involved in the forum. I understand in some chat forums, he was still, or still is active, I'm not sure.
Bob Therriault
21:58-22:25
Yeah, we should get Dan on because definitely I think the really interesting, and it was a thread that I sent to everybody else from the forums, was the point at which he found that you could actually create a verb that could create all the parts of speech. and then it was immediately blocked. Because I think, as Marshall pointed out to me, it was a bug, but it was a bug that allowed you to do something that was pretty powerful.
Marshall Lochbaum
22:26-22:28
It was pretty far from immediately blocked, though.
'Pepe' Quintana
22:29-25:49
Essentially, that bug allows you to take verbs, not only verbs, but all the modifiers, adverbs and conjunctions, as arguments of tacit verbs and produce anything, not only nouns but now verbs, adverbs, conjunctions. And not only that, but you can now, with that bug, it's a lot easier to produce adverbs and conjunctions tacitly. And you don't have, if you see any complex adverb and conjunction is coded with explicit what they call JTRAN, it looks more like a Fortran explicit things. You don't have to. With this, it's just, to me, a pleasure. For example, you want to have a bunch of verbs, the main things, and this is what this code is doing. Each line in this Turing machine, each line represents a verb. And then it's just a pipe of them. They are composed with an at and a colon. each wants to accomplish what it needs to be done. And if you think about it, it would be like at slash, like putting ats in between each verb. It's like... It's like the operation of that, but not as regular operation, but now a conjunction. But that bug allows you to do that, to see conjunction as a verb, and then you can do it. It's like magic. That's the main reason I split, because we do that. Adverbs and conjunctions, we program it that way. You don't have to for adverbs. And now you can do also conjunctions. Without doing that, that is the hard way. Dan also realized at some point that you can do any adverb tacitly. But it was difficult. The way to do it in general is if you want to produce something, even verb, adverb of conjunction, well it has to have an atomic representation, right? And a verb can produce any atomic representation. The only thing... is that you need an adverb to execute a verb to be able to produce any figure of speech. Nouns, verbs, adverbs, conjunctions. So you can do it even following the rules, but it's hard. It's very hard. Dan Bron did a lot of the base of that, but I put it together. And it's in the forum. It's called hg. And it's an adverb that allows you to produce any figure of speech the hard way. But it's, as I say, When you can verbalize or you are verb in adverbs and conjunctions, you can use tacit. All the things that you can do tacitly, you can use it to produce that.
Bob Therriault
25:49-26:00
Does that mean that if you were able to create all the parts of speech from verbs, does that start to open up the door for first order functions? Is that what the risk is there?
'Pepe' Quintana
26:01-27:45
When I was playing with that, there was a huge controversy, and then it says verbs shouldn't produce verbs, for example. And why not? And a lot of the answer is, well, it was not allowed in the original language, so Iverson should have had his reasons. Unfortunately or fortunately, I always question authority everywhere. I try to see what you can do. And to me, you can do a lot of things a lot easier that way. It's not only that. It seems that the core at that time, at least in J804, started changing when the Roger changed and things. [04] But there was certain structure that allowed you to do that. And all, for example, all the structural verbs work with entities. So you can box a verb. You can even quote colon format. If you pass a verb like that, it will give you the linear representation. I don't know how things are done in the interpretive, but it seems that everything has the right structure to do all the things that you want. Like, as I say, all the structural primitives work, even if you pass. The thing you have to do it tacitly, because I believe they say, well, we don't allow nouns, so we forbid that. So in explicit functions, I think there is some line at the end. Marshall can prove me wrong if it's not the case that it's not noun, produce an error. So you are not allowed.
Marshall Lochbaum
27:45-27:48
Yeah, I think that's about what happens.
'Pepe' Quintana
27:48-28:20
In tacit, there is nothing like that, because it can never be passed to a tacit verb, something that is not a noun. So we don't have to put that guard there. However, again, there is a conjunction. There is a backdoor. And Dan half discovered that, but didn't realize how far extended. You can write a verb that produces any other word, not just nouns.
Bob Therriault
28:20-28:22
Why was it considered a bug?
'Pepe' Quintana
28:22-28:35
Because they say verbs can only produce nouns. It's a little tricky because if you see the dictionary from the beginning, it says they produce nouns. It doesn't say, andonly nouns.
Bob Therriault
28:35-28:54
We'll put a link of the discussion in because there's actually one of the comments, I think it was you that made it, said, well, if you wanted to say you couldn't produce anything but nouns, you should have said anything but nouns. But you didn't. You just said they would produce nouns. That leaves it open. So it was just convention then that they didn't want to go that direction?
'Pepe' Quintana
28:55-30:42
If you write code directly, you can go along with it. You don't need that. But if you write programs that write programs, and that is adverbs and conjunctions and verbs, then you can do a lot of things. Because again, the programming adverbs and conjunctions is reduced to verb programming. if you take it that way, and tacit programming is complete, so you can do everything tacit. Again, you can do it the hard way, even with the convention. And we figured out, I think, between Dan and I think Raul helped a little bit. Raul is not a tacit fan, but he's good at it. Anyway, he said that... then you can put the whole thing together and you can have an adverb that takes a verb and produce an adverb that takes the argument and executes the verbs and gives you the answer. So if the verb produces an adverb and conjunction, you are right there. You don't need anything else. The culture way is to produce an atomic representation or equivalent and then use evoke, colon six, train, to produce the final figure of speech. Yes. And we did it when many of the modifiers were taken away. And you can do only with adverbs. putting one adverb after another, and the adverb conjunction. I think they were the things that were left, but that was enough. Produce any figure of a speech with an adverb.
Bob Therriault
30:42-30:51
And the big advantage you've got when you're tacit for how you're using it is that you can write programs that write programs. Is that right?
'Pepe' Quintana
30:52-33:00
Easily, yeah. I have my reasons for doing everything tacitly. Maybe I can give you briefly some of those, because some people say, "Are you crazy? What do you want to do all the tacitly?" One reason is because it's possible. Again, the Turing machine shows it's possible, not only that, but it's practical, in a way. The other thing is when you fix it, you don't have any names, so you don't have to worry about using libraries from somebody else, because if you have no names, there's going to be a name collision with other parts. That's a big advantage. You might criticize and if you see that code in the Turing machine, well, you're using names there. Yes, we're using names there, but only affect the functions that you are defining. After that, you fix it and you are home free. There is another important reason in finance and you have strategies and things. People steal models. So security is an issue. And J is open. But if you fix it, it's not that easy. I remember Roger one time was visiting us when we were in Hoboken. We have a very large tacit verbs as raw primitives, but we wondered if we will hit a limit. Well, some can be hundreds of thousands of characters. Can I have a look? I said, well, I can show you part of one. And I put it in the screen and he opens his eyes and he said, oh, this is the best software security I have ever seen. And he's right. If I lose the one that generated the tacit verbs, I will have a hard time. So we protected even doing more things to scramble the code, but you cannot have a full security, right? If you scramble things, you can always find a way to unscramble.
Bob Therriault
33:00-33:17
And in J, you have the adverb for fixing things. [05] So it's just f dot. is all you require and you feed that verb to it and it fixes it so everything locks in. So would that be sort of equivalent to like a closure? Is that locking everything in so you don't need to worry about where you transport?
'Pepe' Quintana
33:18-36:12
Well, the point is that in this case, once you fix it, it has no names. That was one of the things I needed to be able to bring code and execute it and originally J didn't have that and I asked Roger I sent an email and said wouldn't be good to add this to J and then Roger the businessman says um yeah will you pay for it? I said well maybe let me check and we pay for it. It was put not as a primitive external facilities, the 128 bang colon two I believe. And that allows you to bring code and to execute it dynamically. And that's good because we have this system and we have modules and we bring the fixed code as the module is executing and produce results that are saved and passed to another module that works that way. So with big modules that are independent of each other, you can even modify a model when the system is running as long as... If it happens that it's a model that you're executing, well, it will finish until done. But the next time it will produce the new one. So it's very, very good for maintaining. So once it's fixed, that's it. You can use it with no fear. We use some libraries that are produced. And in that sense, it's not fully tacit. It's fully tacit except for the, we don't want to recode everything, right? Big part is the... some technical libraries that, I mean, you can do it in J, but it's more efficient to do it in something else. We at some point, we need to calculate a eigen decomposition of a matrix, the values and vectors. So we use one of the libraries we call it. Also, a lot of the data is stored in databases. So we need like SQL interface that some have developed. But again, we call it from that. We have something that is external and loads whatever needs to be loaded and executed. The way we do it is we have our own locale and we mind our business there. It won't have any names anyway. It's just executing things there. And then all the libraries are loaded in the normal way. And this external gives the name and executed as an external function. Did that kind of answer your question?
Bob Therriault
36:13-36:19
I think so. The original question was about closures. And I don't know. I don't know. Marshall, do you have a...
Marshall Lochbaum
36:19-36:53
Well, I mean, the main point of a closure is to... You can create closures from a given function in the source code as many times as you want. And each closure you create has a state that it carries with it that can be different from closure to closure. And it's mutable. So, I mean, provided your language can update variables. So the point of a closure is kind of to, it's almost like objects. It's to create as many... different values as you want that are keeping state with them and can update it. So that's pretty different from tacit fixing.
Bob Therriault
36:54-36:56
Yeah, because tacit fixing is like you're fixing the function.
Marshall Lochbaum
36:57-36:57
Yeah.
Bob Therriault
36:57-36:58
But not the...
Marshall Lochbaum
36:58-37:37
Well, and also the fixing in J is a little weird because the reason it's necessary is that J actually passes its tacit values by name in some contexts. So like Dyalog and BQN don't... Like if you define a train and you use a function inside it that's also a train, the final value doesn't know the name of that function that you used. It just expands it to a train. It uses the value of that function. Where J actually keeps the name around, and as I understand it, the reason to do this is because it enables tacit recursion. So to tell it don't do that, then you have this fix adverb.
'Pepe' Quintana
37:37-41:34
There is one thing with tacit related to what Marshall is saying. And is when you are coding this way, the main workhorse is at colon, is the at, right? And it's composition of functions. In general, or composition of verbs in this case. And within a function, the state, if you call it that way, is passed one function to another, the whole thing. And it's updating and doing what it needs. So at some point you can freeze the state, for example, saving. And we code a lot of facilities to debug. First of all, you have to find the line when it's breaking. But just as this can produce decomposition, you can modify the verb in any way you want. for tracing. So you can display which function is executing as it's doing it. So you run it that way and it will tell you which line it breaks, the previous line that was before the breaking. And then you can also save the state at that point. and then explore more what the problem was. And again, when you are doing things, you can pass a state and a function to execute. So I guess it's similar to closures that you pass the data and how it will resume that way. Do you have any other questions? I can tell you a little bit of the Y combinator in J, even if recursion was not available, but not only for verbs, but for adverbs. That's also what's key, because you would like to have recursion, the ability to call recursive adverbs. and conjunctions. [06] And that is not possible without using a name in J. You have that dollar colon for verb, but you don't have anything for adverbs and conjunctions. So to write tacitly an adverb that will allow you to produce adverbs and conjunctions legally, you need recursion. At that time, before the reintroduction of the old modifiers, there was no way to produce conjunctions by the user. You have the given conjunctions, but you can produce adverbs with another adverbs, but not conjunctions. So you could have adverbs that take an argument and produce another adverb that can take another argument, and you keep going like that, similar to, I understand, the lambda functions work and get several arguments. So an adverb produces another, takes another and another. Actually, kind of a strand notation. The one that discovered that was Dan Bron again. He did that, but then of course he wrote that adverb explicitly. So with this J-combinator, you can call adverbs recursively, and I can do this emulation of a strand notation using the Y-combinator, and adverbs that produce adverbs. And you need an ending. For example, if you either know the fixed number of arguments that you are getting for the adverb, or you put a guard at the end, I use left bracket colon Cap. I never use Cap. I don't like Cap.
Bob Therriault
41:34-41:35
Adám doesn't like Cap either.
'Pepe' Quintana
41:36-43:03
So I use it as a guard. Anyway, I put it there. So you can get all your arguments. You can have... adverbs that take many, essentially any number of arguments, and then do whatever you want. That was important because at that time, as I said, you couldn't produce a conjunction, but you can produce what some call double adverbs, that you can take two things, and it's equivalent to a conjunction, as even more general, because you can have many adverbs. In our system, we can produce conjunctions directly. But even then, I was kind of arguing that you just needed one, essentially one, but two of the old modifiers to have conjunctions and to able to produce tacit conjunctions. And that was the form conjunction adverb, that's the core. But to be a little nicer, you would like to also have adverb conjunction adverb. Those two things will help, even just the first one. And they told me, "Forget about that. Those things will never come back." Never say never if they came back. But anyway, that was too late. We figured I had to do it without them.
Bob Therriault
43:04-43:18
Yeah, it was actually an episode of the ArrayCast that Henry told us it was never coming back. And then I think it was within six months he looked at it, and it was more work to redo all the documentation than it was to bring them back. So he brought them back.
'Pepe' Quintana
43:18-43:23
Yeah, that's a problem.
Bob Therriault
43:23-43:45
If somebody wanted to get into tacit programming, you listed a lot of the... advantages of it and things. What would you say is the best approach to getting into it? You've got a really strong mathematical background. I think that helps. But is it just getting in and trying things and gradually getting more comfortable with it? How would you encourage somebody to look at tacit programming?
'Pepe' Quintana
43:45-44:14
Well, my experience is I play with J. And I discovered this task. It was intriguing. I didn't understand at the beginning. It was similar to APL. It's a shock. What is this? How does it work? They were doing all these things, sometimes compact and very nice. So there was something called 11 space colon, I think. Now it's 13 space colon. It's a Tacit Translator. That was my teaching.[07]
Bob Therriault
44:14-44:15
That's 13 colon now, yeah.
'Pepe' Quintana
44:16-46:45
13 colon. It used to be 11 colon, I believe. And that was my teaching. I saw something, simple functions and said, well, how can they do it? I tried it, I couldn't figure it out. And then I would try the translator. Ah, then you can use left bracket, right bracket, put some arguments, what you want. You can use forks to combine them. So for a tacit, I would say forks are very, very important. and the other is the at from my point of view, the composition, that's the core. And the composition, I hope I don't get in trouble here, but some people talk about functional programming and all that, and they can use category theory to model these and get results. And composition is the key, right? Because it's associative, you have a neutral element that is the right bracket, the identity function, and you can do a lot of things with that. Essentially, that's what we are doing. We are composing. And you can use that for certain things actually, because the problem is when you want to pass transparently some extra information and you don't want to modify the code drastically. So you can use some ideas of my thinking, I have done it, to instead of go and mess with that, rewrite ad that they will pass those things along the way. I guess it's similar to closures. I think that's the idea with closures, that you can pass those things and it's almost transparent. If you change the composition to pass the information that you want. For example, if you... One, you are using a recursive function and you have a work and then, oh, I want to count how many calls are, but I don't want to modify the function directly. So you can change the composition, how these things are passed in a way that is almost transparent. Does that make sense? But as you said, maybe I'm not sure if I convoluted that.
Bob Therriault
46:46-47:00
Yeah, I'm not sure I quite got that. I think until you went sort of sideways and then said that you were using it composition within, but you wouldn't do it within the function. But I guess it's outside the function that you're defining. It's at the point that it moves to the next one.
'Pepe' Quintana
47:01-47:10
Yeah, you redefine. You redefine. By the way, in this code that you see, I use little o as composition directly.
Bob Therriault
47:11-47:11
Yeah.
'Pepe' Quintana
47:11-47:58
And that's the one that you can redefine. And as I said also, in this case, each line is really composing those birds, represent And is the composition of those verbs. For example, if you see the definition of the Turing machine displays the last verb, well, it's the one that executed. Last is the first one appearing if you want to put it as composition, right? Is this play and then the other and the other. So it's good in that sense. It's very easy to modify, as I say, for tracing, for debugging, but also for implementing other things.
Bob Therriault
47:58-48:13
You see, I always thought you just replaced your at colon, your at, with... the small, oh, because it looks nicer. But actually when you redefine it, then you can do other things with it too. And that does make a lot of sense.
'Pepe' Quintana
48:13-48:39
Right. And there is an implicit at there. The is there, it's that container, but I use this controversial way to produce more than fairs and all that is a quick and dirty, tacit, what you call wicked tacit. But the is, is an adverb, right? That is producing the tacit there.
Bob Therriault
48:39-49:04
Yeah, I think Dan talks at one point in one of the discussions about the adverb trains and how he, in fact, when you mentioned 11 space colon, in one of his discussions, he actually talks about the fact that changing a series of verbs to an adverb train is actually pretty mechanical. He says, maybe I should look at creating an 11-space colon.
'Pepe' Quintana
49:04-51:33
Well, yeah, yeah. And then he wrote something, and he didn't use recursion, but it's a lot easier to write. to do it recursively. Essentially what that does, you have a text that represents some code in J, and then it's transformed in some form, not necessarily in gerunds, but some expression that when you use train the back colon 6, it produces the the item that you want. And that's very powerful, actually. It's one of the things that we use when you have a very tough code that you want to translate. The tacit translator works with simple things, and also it tells you you cannot have things that are interacted with modifiers. So the tacit translator is not going to help. But on the other hand, basically you have an expression there and you want to replace the arguments in the right places, right? And what Dan did it almost does that. So I took that, I recorded it recursively in a way that is fixed. That's one of the problems, let me put a little parenthesis of J. Anybody that writes and verbs recursively in J has a problem sometimes with the scope. [08] You can have your very nice tacit recursive function, and then you embed in another tacit verb, and it works. And you fix it, and it doesn't work because the scope changes. So it has an implicit scope that gets in the way when you are fixing it. So it should have had scope, at least to put it when you need it. And that's one of the things we added, but actually the tacit toolkit that runs in J06 and earlier, even J07 I think it runs. It allows you to do that if you are using these illegal, non-culture ways to produce adverbs. You can put an adverb and you can put your code and it will be the scope for that and you can fix that and it won't interfere anymore.
Bob Therriault
51:33-51:39
One of the things I really liked in the discussions is you term all those things as wicked. The wicked tools.
'Pepe' Quintana
51:39-51:41
Yeah, wickedly because it's like black magic. It's not quite magic.
Bob Therriault
51:42-51:45
Exactly.
'Pepe' Quintana
51:45-55:27
Again, going back to your question, I learned from that a lot. And actually, the idea of modifying boxes little by little came from that. And in one thing, they had more arguments than two. And it is something like that. It used it as a function. It was not very good because it was recalculating everything. I think in that presentation of the conference, I mentioned that you can do it like that, but it's not efficient. And practically, it went wrong. But with boxes, it's very efficient when you amend something. You are not passing the whole thing. You are just... working with the boxes to be changed, and they are pointers on that. As I recall, Marshall was the one that told me that, "Oh yeah, you have no problem with this," because that's the way J works. We added some things, a part of the apply. Another thing we need a lot. We have some sequential, the Bayesian implementation of the models that we have. You do it sequentially and you update parameters sequentially, but for simulations you want to have history to produce portfolios and to test them. So you are operating some inputs that refresh your hyperparameters of the Bayesian model, but you want to keep history. What I'm talking about is fold. I need fold. I didn't know it was called fold, but eventually I realized that's what it was. So Roger came and implemented fold. And it worked for the things that we wanted. They were kind of complex, but strangely, we found that for some simple cases, special cases, it had a bug. Yeah. And we didn't know how to fix it, but Marshall here fixed it for us. He was doing an internship, and one of the things he did for us was to fix that code. And we have a fold, essentially, but a simplified version. To me, the fold that is implemented is great, but it has too many bells, which is... The one that we have is small, and it was implemented directly so that the things run a lot faster. I have a version. I didn't that. default but is appending is a problem to keep the history we wanted to keep the history of the state so other things that Roger did was that he teached somebody that then is no longer with us but then was able to do c code and he was very extremely professional he didn't hide anything he knew that if he teaches us how to do it, we might not call him as often, but my respects to him. Royer came, I don't know, two, three times. We talk about many things. And I feel really, some of you were expecting. I was not expecting when he passed away. And I saw people in the forum saying things about Roger. I couldn't even bring myself, even now as I'm talking, I'm getting a little emotional. I literally, I have no words.
Bob Therriault
55:28-55:42
Yeah, Roger was a really special guy. We did an episode with Morten, I think, early on, just after he passed away. It became apparent how important and important. like into the languages, but also as a person. He was a really incredible man.
'Pepe' Quintana
55:43-56:05
Yeah, as a person he was. As I say, he was so honest. However, we have extended because you can extend J by at least putting some J code and producing, I think, and they call it magic. I don't know how it is. Sometimes I get involved just to be cool.
Bob Therriault
56:06-56:15
So, Conor, the ocean of Tacit is very deep. Do you think you want to go back into it again?
Conor Hoekstra
56:16-57:56
I mean, I've never left Tacit. Yeah, I'm a little concerned that Pepe's top three reasons to Tacit program were... it's possible number two was you don't need to have names and therefore you can work with other libraries and three he basically said it's code security I've heard of the quote job security through code obscurity I've never heard of code security through code obscurity And I actually, I went and I took the dense, tacit version of the universal Turing machine, and I fed it to Gemini and just said, what is this? and And it did have UTM in it, which was a little bit of a hint. And it did well. I mean, it's trained on the whole of the internet. And Gemini 2.5 said, that code snippet is written in J, a high-level array-oriented programming language. Et cetera, et cetera. And then the last paragraph, it's only about four paragraphs long. Yeah. The last paragraph says, this snippet you provided is a classic example of this style. It's almost impossible to read without a deep understanding of J's idioms as it relies heavily on the tacit style and a dense arrangement of operators and numbers. So I'm a bit surprised that this was the direction that this tacit episode went, but I've never heard an argument, like there is the discussions of readability versus unreadability. I've never heard for someone advocate for the style because it is unreadable. I'm sure you mean that in the sense that if you densify it that much. But yeah, I'll let you respond if you want.
'Pepe' Quintana
57:56-01:00:33
If you allow me to clarify some things. If you don't know how it was built, can you just put that? It's very hard, especially when you have a very, very large tacit code, like hundreds of thousands. However, tacit to me is a lot easier than explicit. As I said, the... tacit translation was my teacher, but I became so embedded in tacit that the explicit started to have difficulties, and I started using the other way to understand the not tacit code. I'm not very good. I program by ear, if you allow me an analogy. I don't know all the parts, instructions of J, but I see patterns and that's the way I program. Sometimes I don't get it and fix it a little bit. But the J interpreter parts everything for me. Especially I use a lot the boxed display of verbs and other things, right? [09] And if I see the box displayed, immediately I can see what is happening, how the data is flowing to all these forks, verbs, adverbs, whatever we see. It's so clear. Whereas if it's regular, non-tacit, I have to remember the rules and it goes slowly. So to me, it's a lot easier to do that, particularly because of that facility. It says there, or it used to say, I don't know if it still says, oh, you can use this if you are learning J, or implying that experts want to do. Well, I was introduced as an expert, but I am still using that all the time. Well, not all the time, but any time I want to understand. Even something that I wrote, I want to say, "Oh, what I was doing here?" And then I see the display representation and it's clear. Once you get the basics of tacit, then that helps you a lot to see the flow. And sometimes you write something that doesn't work. You do that, ah, is that this is binding in a different way, I wanted to bind that, or I need extra parentheses here, or whatever you need to fix. That probably is one of the most important hints. that I can keep.
Bob Therriault
01:00:33-01:01:04
And in J, when you type in the name of a verb, it replies, it shows you the verb. And the boxed version has the advantage of it embeds boxes in such a way that you can tell the precedence and the process of the data flowing through the verb. If you do linear, it just lays out the verb. That's not as useful unless you already understand the parsing. But the box actually gives you a... that box display version gives you a way of looking at it, and it almost parses for you if you understand what you're looking at.
'Pepe' Quintana
01:01:04-01:01:10
Exactly. I have that. There is a way to turn on and off the representations that you want.
Bob Therriault
01:01:11-01:01:13
Yeah, I think it's a foreign conjunction. I think it's five.
'Pepe' Quintana
01:01:13-01:01:18
Yeah, it's a foreign conjunction, and you have, I think, five, two is what I use.
Bob Therriault
01:01:18-01:01:18
Yeah.
'Pepe' Quintana
01:01:18-01:01:37
And five is the linear representation, and I think two is the display. Maybe I'm wrong about two, but... I have that on all the time. So when I tap a verb, it shows me the verb and the boxed or displayed represent. The displayed representation is based on boxes.
Bob Therriault
01:01:37-01:01:41
And I think there's an option you can do parentheses as well, but I don't think that's as useful.
'Pepe' Quintana
01:01:41-01:01:55
Right. The boxing represents the parentheses and that's why... It helps a lot to see the flow. And that's the thing about tacit is controlling the flow of the data.
Conor Hoekstra
01:01:55-01:02:23
Well, we will be sure to put a link to, even though you said it was a bit old, the Jx 1.1. Because it does, even if you're not going to check with the executable, it does have a guide that mentions the extensions. And there's also a cheat sheet there for folks that want to go check that out. And yeah, if folks have been listening, if they've got questions for Pepe or for us and want to deep dive more into Wicked Tacit, you can reach us at...
Bob Therriault
01:02:23-01:02:53
Contact at ArrayCast.com is the way to get in touch with us. And this may generate some questions. There's a lot of the old discussions we'll have links into. And it's really fascinating to see because I haven't seen... the dive into that area quite as much recently in J. I guess because maybe it's because it was blocked. But it is interesting to see people exploring those areas. And yeah, it was great to have you on, Pepe. And it's wonderful to be able to talk to you. Thank you for coming on.
'Pepe' Quintana
01:02:53-01:02:55
Thank you for having me.
Conor Hoekstra
01:02:55-01:02:58
Yes, it's been a blast. And with that, we will say happy array programming.
All
01:02:59-01:02:59
Happy array programming.
Music
01:02:59-01:03:13