Category: AI

“Hey AI, build me a website.”

It’s a matter of time – probably months, before we get here. I’m surprise we’re not here already:

Prompt:

Build me a website based on the documents and images in this Google Drive folder. Make it accessible and usable on all devices. Make it an app users can pin to their phone. Add a text-to-voice feature on every page. Pull in my videos from YouTube and TikTok. Style it to be modern and make the styles easy to change later. Give it a robust back-end API for all the data. And an RSS feed for all users. I want to be ready for the future beyond rectangular screens. Please.

If I was in any product related to web hosting or code, I’d be building this right now. I’m sure someone is. I’m sure many are. I’ll be roughly -100% surprised when this drops from a major cloud provider or version control service or CMS within the next few months. Actually, I’d be shocked if it didn’t.

All the automations and frameworks and templates and tools already exist – have for a long time. We’ve been using them to build the modern web. All the back-end hooks and CLIs and CI integrations to do this with any major framework, using any major build process, served from any major host, with any integration, already exists.

All that’s needed is a sprinkling of AI for harmonization; to boot up a repo, spin up a virtual dev environment, pull the necessary dependencies, set up the configs to match user specs and keys, configure the setup, populate the components and styles and pages, run the build, and push it to the host.

And once the project is built, in-browser live visual editing with immediate code updates is also already here.

Builder.io has been close for a while. Cursor and Replit are almost there. Others are hot on their heels.

This isn’t a future prediction—it’s a future certainty. We are already here, we just haven’t realized it yet. Because nobody has pulled the pieces all the way together. Yet.

We built our processes for this moment – we just didn’t realize it.

Building modern websites required knowing frameworks and build processes and optimization and CI and serving strategies, and knowing how to put them together and make them sing. That was where developer expertise came in; In the cryptic knowing and intricate wiring together and alchemic configuration of the pieces to make up the whole. That was the gate. And we kept it closed by building more complex and streamlined developer experiences.

This, through a different lens, is AI project management. This is already possible – it just needs an interface and the right permissions.

This is what’s next. And for the web dev industry, it’ll be upending, revolutionary, a transmutation.

This is also coming for every other software dev field, it’ll just take a bit longer.

Devs will still have work, plenty of it. It’ll be different work. More demanding, and interesting, and challenging, and human work.

When AI makes coding and project management cheap, when throwaway software becomes ubiquitous and sufficient for the task, when one-size-fits-all becomes one-experience-for-every-user, the value of a skilled developer shifts – from coding and management to product design, and accessibility, and internationalization, and user experience, and data management, and privacy, and security, and serving the user the data and interaction they need, in their context, in their moment, at their convenience and control.

And all this at the end of the web as a screen-based platform where the same website serves every user.

Are you ready for what comes next?

—

Cross-posted to LinkedIn.

Asking users to opt-out of AI training is a deceptive pattern. Governments and regulators must step in to enforce opt-in as the mandated international standard. In my opinion.

In May 2024, European users of Instagram and Facebook got a new system message informing them all their public posts would be used for training AI starting June 26th. To exclude their content from this program, each user (and each business account) would have to actively opt-out – a process that requires knowing where to go and what to do. Additionally, even if you do opt out, and even if you don’t even have a Facebook account, Meta grants itself generous rights to use any content it can get its hands on for AI training. From their How Meta uses information for generative AI models and features page:

“Even if you don’t use our Products and services or have an account, we may still process information about you to develop and improve AI at Meta. For example, this could happen if you appear anywhere in an image shared on our Products or services by someone who does use them or if someone mentions information about you in posts or captions that they share on our Products and services.”

Bottom Trawling the Internet

Meta is not alone in this. The established standard for acquiring AI training data has been to scrape the internet of any publicly available data and use it as each AI company sees fit. And as with bottom trawling, the consequences to privacy, copyright, and the livelihoods of many creators are severe.

Historically, AI scraping has been done by default, without warning or even acknowledgement, often as part of general web scraping to support search indexes. As awareness of this practice has grown, some companies like Automattic (WordPress.com, Tumblr, etc) and now Meta now offer opt-out features so users can exclude their content from AI scraping, but this often comes with direct consequences to visibility and functionality. My cynical hunch is the platform companies are aware of the public pushback around these practices and they are now covering themselves legally. My hope is platforms offering an explicit opt-out potion means they have realized the wholesale scraping of the web is ethically problematic and they are at least trying to do something about it.

Here’s the thing: The opt-out is part of the problem!

Power and the Principle of Least Privilege

A few years ago I attended a conference where each attendee was given a choice to attach a black or red lanyard to their badges. Black meant the event had permission to take photos and videos of the attendee, red meant they did not. If you didn’t choose (or like me didn’t listen when it was explained) they gave you a red lanyard.

This is a real-world implementation of the Principle of Least Privilege: Photographers were only allowed to create images of people who gave explicit permission; the attendees who opted in.

At a different conference that same year I saw the reverse of this approach: Scattered around the venue were posters reading as follows:

“The [Conference] reserves the right to photograph any attendee for use in promotional materials. If you do not wish to be in the pictures, please notify the roaming photographers.”

Here, the attendees were opted in by default, and it was up to each attendee to actively opt out at each interaction with a photographer. Needless to say this is not feasible, and as a result everyone at the conference either relented to having their pictures taken or left.

I think most will agree the first conference acted ethically towards the attendees, the second did not. In fact, the second conference experienced a major backlash after the event, and the following year they handed out “NO PHOTO” stickers for attendees to put on their badges if they so desired.

There are two important takeaways here:

First, when it’s a real-world situation, most people immediately see the ethical missteps of the second conference. And second, even so most attendees stayed at the conference knowing they might be photographed against their will.

The conference created a power dynamic where people who didn’t want to be photographed were left with bad options: Constantly be on guard for photographers to tell them they did not want their picture taken, or leave the conference they paid and probably travelled to attend. It’s unethical, but it’s not explicitly illegal, and in the end it means they get more promo shots to use. So be it if some attendees are uncomfortable.

AI scraping and the current opt-out strategy falls squarely in the same category as the second conference. While the obvious ethical choice is to let people opt-in to AI scraping, an opt-out option provides just enough cover to not get sued while ensuring broad access to content because most users won’t go through the trouble of opting out – especially if you make the feature hard to find and hard to use.

My Content, My Choice

Platforms have long argued they can do what they will with user content. In fact, using user content to meet business needs is the economic basis for most platforms, and this is the bargain we’ve collectively agreed to.

Building on this premise, platforms and AI companies now want to extend this principle to AI training, claiming both that they have a right to use the data without explicit permission because it’s public, and that not being able to use it without explicit permission would make it impossible for them to operate at all.

I think it’s high time we to question both these stances:

Letting platforms do what they wish with our content was always a Devil’s bargain, and we’re now acutely aware of how bad of a deal it really was. The negative effects of surveillance capitalism, filter bubbles, and ad-driven online radicalization engines (nee “recommendation algorithms”) are plain to see and play a significant part in the erosion of everything from privacy to democracy.

The claim that an entire business category can’t be competitive unless it has free access to raw materials is one we’ve heard before, and again we know the consequences. Bottom trawls and overfishing has depleted our oceans, pollution chokes our air and waters, the exploitation of cheap labour in the global south keeps billons of people in chronic poverty. To say these are false equivalences is to ignore the reality of what we’re talking about. While the actual bits and bytes collected during an AI scrape are not a finite resource, the creative energy that went into creating them are. And the purpose of scraping data from any source is to train a machine to mimic and otherwise use that data in place of a human mind.

Opt-out is a slow poison because it puts choice just far enough away that becomes out of reach for most people. It makes a choice on our behalf and then forces us to negate it. It’s exactly opposite of how it should be.

The Choice is Ours

We are at the very beginning of a new era of technology, and we’re still figuring it all out. This means right now we have the power to make decisions, and the responsibility of making the right decisions.

This is the moment for us to learn from our mistakes with surveillance capitalism and take bold steps to build a more just and equitable world for everyone who interacts with technology.

One of the first, and most straightforward steps we can take right now is to make a simple regulation for all tech companies dealing with user data:

Users must opt-in to any change in how their data is handled.

And to protect users:

Choosing not to opt in must not impact the user experience of existing features.

This puts the onus on the AI companies to get consent when collecting data to train their models, and gives users agency to choose what if any AI training they want their data included in.

If I wanted to make a name for myself in the political realm, this is where I’d start: With a self-evident regulation protecting the rights of every person to own their own work.

We shall see.

---

Originally published in my newsletter.

Is OpenAI imposing a token tax on non-English languages? The token count on international languages are significantly higher than English. This is a serious accessibility and equity issue for the global majority who use languages other than English.

I did an experiment using OpenAI’s Tokenizer: Get the token count for the same text in different languages, adjusted for character count. Here are the results:

• ?? English: 105 tokens
• ?? Spanish: 137 tokens
• ?? French: 138 tokens
• ?? German: 138 tokens
• ?? Dutch: 144 tokens
• ?? Norwegian: 157 tokens
• ?? Hungarian: 164 tokens
• ?? Arabic: 286 tokens

For ChatGPT users this means international users will hit the token window limit faster resulting in higher risk of “hallucinations,”, reduced response quality in longer conversations, and reduced ability to process larger volumes of text.

For OpenAI API users this also means significantly higher cost for every prompt and response as they are charged per token.

For English, the tokenizer counts most words as one token. For non-English languages, most words are counted as two or more tokens. This is likely because English dominates the training data GPT was built on, and results in an effective token tax on non-English languages, especially smaller and more complex languages.

As AI becomes an ever more present component of our lives and work, this token tax poses a significant equity problem disadvantaging the global majority and anyone not using English as their primary language.

“One of our persisting challenges is excessive hallucinations.”

I’ll cut right to the quick: AI systems are nothing like human beings, but our language makes us think they are. That’s a problem, for us and for the AI systems we’re building.

When ChatGPT was introduced to the public in November 2022, people were baffled to discover when you asked the system a question it would often return text presenting information which appeared to be true at first blush but on further scrutiny was only partially true or even entirely fabricated. A barrage of news articles and social media commentary followed, about how the AI systems were “lying” or intentionally “deceiving” us, suggesting these passive computer systems were acting out of some form of malice. 

AI experts explained this is a well-known phenomenon called “hallucinations.” The term quickly took root in the public consciousness and provided a platform from which our common understanding of these technologies would grow:

“If a machine can hallucinate like me, it must have a mind like mine.”

A relatable metaphor can be a useful way of explaining something complex by referring to something similar and less complex. Metaphors are found throughout our everyday language: “You are an angel for doing this”, “she was on fire today,” “I am toast,” these are all nonsensical statements with real-world meanings easily understood by people with sufficient language skills and shared cultural and societal experience. 

Metaphor can also be a useful tool when explaining complex concepts without requiring the listener to understand the full complexity. When our son complains about his hands hurting after he’s been coloring for a long time and I tell him it’s because they are “tired” he understands he needs to give them a “rest” without needing to understand the physiological causes of muscle fatigue. When a TV show is abruptly interrupted by an error message on the TV and I tell him it’s because our TV can’t talk to the streaming service, he understands this is a communication problem without needing to understand the intricacies of HTTP protocol, DNS servers, or packet loss.

So when seemingly all-knowing AI systems inexplicably fabricate information, it’s easier to explain what happened through the metaphor of hallucination than it is to explain the inner workings of computer systems even the people who build them do not have a complete understanding of.

Throughout the history of the science of artificial intelligence, we’ve used metaphorical language rooted in human cognition and behaviour to explain how these systems operate. The term “artificial intelligence” is a metaphor describing systems whose capabilities go beyond traditional computer systems and are “smart” the way humans are smart. Saying AI systems “learn” about the world through “training” uses education metaphors to make simple the enormously complex machine learning algorithms and processes that go into building their models. Saying AIs have “knowledge,” “reasoning” capabilities, and the ability to “follow instructions” uses metaphor to explain their often surprising power. We use the metaphor of human communication and interaction when we tell people to “have conversations” with the systems and refer to them as individuals with human traits like attitudes and emotions.

When we use these metaphors to describe AI, people get enough of an understanding of what’s going on to be able to speak about these systems and see how they can fit into their lives and work without having to understand their technical underpinnings.

The problem is by using anthropomorphic language – metaphors referencing human traits – we construct an image in our minds of these systems being variants of ourselves: machines that are intelligent like us humans, that learn about the world through training like us humans, that hold knowledge and reason like us, and follow instructions like us, and have conversations like us. And when some of those systems use our own very human language as both input and output, our metaphors get validation in the real world and we start thinking of the machines and their software as living conscious agents even when we know they are not.

No wonder then when an AI outputs information that looks true but turns out to be a fabrication we continue the pattern and describe the machine as a liar. 

We could have, and probably should have, chosen to use more technical language for these machines, but in doing so we’d have missed out on the magic and the marketing. “This is the courtyard and Juliet is a human on the balcony” pales to the evocation of “This is the east and Juliet is the sun!” because the language we use colours and shapes our understanding of the world. So now that we’ve chosen to use human metaphors to describe systems of non-linear computing algorithms that process information and build network models, perform advanced retrieval from data graphs and calculate responses based on neural networks, take input and produce output in the form of tokens, and output statistically correct but sometimes fabricated token sequences that reproduce human language, we must always be on guard against the hallucinations our language conjures within us.

I fear in our attempt to make AI more understandable we have committed an unintentional act of self-deception. The metaphor of humanity rides too close to our dreams of machines built in our image, and our language makes us confuse those dreams with reality.

In the Age of AI, our biggest challenge may be overcoming our own excessive hallucinations.

—

Cross-posted to LinkedIn.

“We’ve trained a model called ChatGPT which interacts in a conversational way. The dialogue format makes it possible for ChatGPT to answer followup questions, admit its mistakes, challenge incorrect premises, and reject inappropriate requests.” – OpenAI Blog post “Introducing ChatGPT,” November 30, 2022

With these words, a transformation – from the time before to our time with the machines and systems we’ve chosen to call “Artificial Intelligence.”

ChatGPT, launched a year ago today as a “research preview” of our future, was our dreams and our nightmares made real: A computer we could talk to that seemed to talk back. A mirror and a marvel, at times indiscernible for magic, brimming with peril and potential.

Born from the transformational transformer, the Generative Pre-Trained Transformer outfitted with a chat interface would prove to be the tickler our imaginations needed after three years of pandemic trauma and looming economic uncertainty.

Can I get this thing to do my work, my homework, my chores, can I outsource my thinking, make it my therapy? Can it be my friend, my companion, my lover, my master?

The fenced path toward our carefully designed future suddenly opened into an infinite opportunity space with us standing at its edge, in awe, uncertain and excited and afraid to take our first steps, take any steps, to go at all because where do you go when you can go anywhere?

Listen carefully in those days to the whispers in the halls of power and you’d find those who thought it had come too soon; this tool that broke the academic confines of AI and put it in the hands of everyone. They had not yet the time to forge our path towards their vision. No fences were ready to keep us in line. Our future with AI was suddenly here, unannounced and unbounded, with imaginary grandmothers telling children’s stories of napalm and search engines catfishing journalists and a democratization and lowering of barriers to almost every field of exploration and creativity and discovery.

“Just a perfect day, you made me forget myself. I thought I was someone else, someone good.” – Lou Reed, Perfect Day, Transformer

One year in and we’ve learned things; about ChatGPT; about the people who built it; about creativity and generation; about people and machines; about work and life; and about us, ourselves, the humans in the mix.

One year ago my colleague asked me to help the world move forward with this. “Show the world how to get this right, how we can get it right together” he said, or something like it. And I said to him “I’m glad it’s us.” And I am. Glad it’s us. Not only in the narrow sense of him and me and our friends who we have the honour and privilege of working with, but in the broadest sense of all. I’m glad we, the global community who share this moment in time, get to explore this opportunity space together, unbounded by prior plans and pre-determined directions.

With AI we have an opportunity to think anew about the things that occupy our lives: How we work, how we play, how we create, how we commune. We get to think again, or maybe for the first time, about our time and how we use it.

We get to ask questions, about the role of technology and the role of humanity, about what we value and how value is measured, about who gets a say as we build our future with AI, and who decides who gets a say. We get to ask questions about power and privilege and access and limitations and what is good and right for us, our communities, and our world.

The decisions we make and the steps we take into the possibility space of AI determine where we go next and where those who come after us get to go in their futures. The time before is gone. This time is something new, and we get to define it.

This transformer in our midst, this up-ender of every process of creation and alteration and organization and interaction, is something so new, so unknown we don’t yet know what it is for. And the way we use it and its myriad of sibling cats well and truly out of the bag and on their way into the world today is the way we used the iPhone when it first arrived in our hands: As a phone with a flashlight app. The AIs brought forward and into our hands by ChatGPT are both tools for us to use and materials for us to work with. With new tools and materials in our practice we get to reflect on our old ways and find new ways – hopefully better ways – to do the old things and the new.

As I stand here, one year into the era of ChatGPT, what I see before me in the near future and several years from now is a period of ceaseless transformation. This year was the preparation; the slow organizing of exploratory teams, the uneven distribution of resources, the first furtive steps on untouched ground. What comes next is the journey into unexplored opportunity space. What we find there and how we use it will be up to us. And if we do it right, if we care for one another and help each other and build paths everyone can follow, it might – no – it will be amazing.

Hope is a catalyst. Build the future for everyone and for yourself.

OpenAI – the creators of ChatGPT and current designers of our collective futures with AI – announced a metric tonne of new updates, features, and products at their inaugural DevDay this Monday. I was there, and here is my no-hype, no-nonsense, pragmatic guide to what was released and what it means for you.

GPT-4 Turbo: Upgraded model with upgraded speed, upgraded “knowledge,” and lower pricing

All software has version releases. So does GPT – the underlying foundation model for ChatGPT and OpenAI’s other language-based AI services. The latest version – GPT-4 Turbo – boasts:

• faster speed. GPT-4 was notoriously slow compared to GPT-3.5 and GPT-3.5 Turbo. GPT-4 Turbo is reportedly significantly faster meaning you don’t have to wait as long for responses. This is especially noticeable with their text-to-speech features.
• bigger context window. GPT-4 Turbo has a 128,000 token context vs GPT-4’s 8,000 – 32,000 tokens. This means you can now provide around 300 pages of text for the system to reference during your session without it losing track of what you’re talking about. LLM systems are language transformers, and the more context you provide, the better they are able to perform tasks. This has big implications which I’ll address in the below sections on GPTs and the Assistant API.
• updated knowledge cutoff. GPT-3, 3.5, 3.5 Turbo, and 4 were all trained on data collected before September 2021. This meant if you asked them about something that happened at a later date, they would not be able to answer. GPT-4 Turbo’ knowledge cutoff is April 2023 and in the DevDay keynote OpenAI CEO Sam Altman said they will “try to never let it get that out of date again.”
• lower cost. GPT-4 Turbo is 3x cheaper than GPT-4 for prompts, and 2x cheaper for completions. This is significant for developers who are building things with OpenAI’s API because every token costs money. This is a transparent play to get more developers to work with the platform. As for ChatGPT, the pricing stays the same, so the majority of users won’t see any pricing impact.
• Other things: Multimodal by default in ChatGPT (Dall-E 3, web lookup, code interpreter triggers automatically), invite-only fine-tuning for GPT-4, and increased rate limits for the API.

What this means for us

GPT-4 Turbo is the next version of GPT, and based on previous history we can expect either a GPT-4.5 Turbo or a GPT-5 in the relatively near future. The model provides incremental and obvious improvements and we see a clear pattern here: Base models improve performance, turbo models improve speed, extend context windows, and lower cost. The real-world implications of this new model are significant:

• ChatGPT will appear “smarter” and more “knowledgable” meaning people will be more inclined to think of these systems as “intelligent” and neutral arbiters of the truth. This continues to be a serious societal problem and will be amplified every time the models get upgraded.
• Using GPT models for practical things got a lot easier. Context window limits have been a major issue for use cases including knowledge retrieval from large documents, summary writing, and more. The enormous context window of GPT-4 Turbo means students can use ChatGPT to summarize academic articles and entire textbooks, writers can use it to review entire chapters and even books, and data professionals can use it to parse much larger data sets.
• More people will be using ChatGPT and GPT-based systems for more advanced things, and lean more on the mythologized “reasoning” within these systems to make decisions, and the systems will produce completions good enough to pass a cursory overview leading people to think they are doing good work. Education is necessary to help people understand why this is not the case.

GPTs: The first step towards GPT Agents and the sidelining of plugins

ChatGPT users can now create so-called GPTs – effectively tailored ChatGPT versions with custom instructions, expanded “knowledge”, and specialized actions. These GPTs are built from the ChatGPT interface and programmed using natural language meaning you don’t have to be a programmer to build them. This democratizes the creation of custom GPT agents and gives people new AI capabilities.

• Each GPT has its own custom instructions – a large system prompt where you describe what the GPT is for, what it should do, and how it should behave.
• You can upload “knowledge” to a GPT in the form of documents and other data and the GPT will refer to this knowledge in its completions. For example, you can upload a textbook as a PDF and tell the GPT to act like a teaching assistant and it can help you learn the content of the textbook, quiz you on important topics, provide summaries, etc.
• Actions allow you to connect GPTs to external services and customize their interactions. For example you can connect a GPT to a weather API and instruct it on how to pull real-time data from that API for accurate reporting.
• You can create private GPTs with any content you want.
• You can share GPTs (when you do they go through a copyright check to make sure you’re not sharing content you don’t own the rights to).
• Enterprise users can create enterprise-only GPTs to share within their orgs.
• There will be a future GPT marketplace where you can buy and sell GPTs with profit sharing.
• Currently GPTs are in beta, available only to ChatGPT Plus users, and being rolled out slowly. Unclear whether they will become available to non-paying users.
• Some mentions were made about how actions could be associated with ChatGPT plugins, but reading between the lines the message is quite clear: Plugins are being silently sidelined in favour of GPTs.

What this means for us

GPTs will become the new primary way people use ChatGPT because they eliminate the need to state the purpose of your interaction with each chat. GPTs will also dramatically accelerate advanced user of ChatGPT because they bring down some significant barriers to entry:

• The massive 128,000 token limit allows you to upload entire books as “knowledge” in a GPT meaning every student can and will create a GPT for every textbook they own and use it to supercharge their learning.
• Sharing of GPTs means as people create new capabilities with ChatGPT they’re able to give those capabilities to others. This will be especially important for things like helpdesk, documentation search, and internal enterprise operations.
• The plugins ecosystem is fading into irrelevancy, both because GPTs take over for them and because the release of GPTs meant the death of hundreds of well-funded startups and projects, all informed by the plugins they created. For example, every “talk to your PDF” type plugin is now meaningless as GPTs do this by default.
• OpenAI will have a nightmare task on their hands as they try to moderate the tsunami on top of an avalanche of GPTs people make and try to sell in their marketplace. Moderation will be key, and it will be enormously costly.

Assistants: The programmer’s path to agents

Along with GPTs (which belong in ChatGPT), OpenAI released the Assistant API which provides the same family of functionality in for programmers who build tools utilizing GPT services. With the Assistant API comes a bunch of features that make the work of every developer a lot easier:

• Threaded responses for streaming so you don’t have to keep track of every prompt/response pair in your own database
• Invoke multiple functions at once with function calling
• “Knowledge” retrieval from documents (low-key low-investment RAG for smaller documents)
• a stateful API ? because this is 2023 not 1996
• API access to the code interpreter (and a future path towards custom code interpreters)
• Future promises including mutimodal API, async, and support for web sockets and web hooks.

What this means for us

If you’ve built any application on top of the OpenAI API, chances are you now have to rebuild it. Many of the new features released (threading, multi-function calling, retrieval, statefulness) replace custom features developers were forced to build due to the lack of core support from the API. This will be enormously expensive and damaging to many projects, but is necessary to move the entire space forward. The lack of these features in the original API were deficiencies and their introduction is long overdue. One thing I didn’t see was any mention of proper authentication. The current key-based auth in the OpenAI API is sub-optimal at best and leaves developers having to rig their own security around their apps which is… not great.

• Building anything with the API is now way easier.
• This is an aggressive play to onboard more developers, and OpenAI is clearly taking slow developer adoption seriously.
• The importance of parallel function calling cannot be overstated – this is the path to a lot of advanced functionality.
• Building extensions to OpenAI’s features remains risky as the API and underlying services evolve, so make sure you have room to rapidly iterate and change.
• I expect we’ll see a continuation of this rapid evolution of API features for a long time, so stay nimble.

In summary

OpenAI is rapidly evolving from a startup with an insanely popular experimental service to a full-fledged platform company with professional products on offer. The rapid evolution of their products shows now sign of slowing down and I expect by next year’s DevDay what was released this week will appear quaint and old.

If you’re working with AI in any way, get used to the constant change and uncertainty, because they are going to keep accelerating for a while.

If we do this right, AI can make our jobs and our lives easier and give us time back to do the things we have reason to value. Pair programming with AI serves as a practical example.

With the realization of AI’s power comes well-justified concerns about how AIs will figure into our lives – and in particular our work. Look to any media outlet and you’ll find a dense fog of articles, videos, podcasts, and think pieces about whether, when, and how AIs will take people’s jobs, and whose jobs are most at risk right now.

In this darkness, let me put up a bright beacon on the horizon of possibility and give you a glimpse of what a future of human-AI collaboration can look like.

Explain it to me

You bump up against a problem at work: an Excel formula you’ve forgotten, an inscrutable data processing script written by people no longer on the team, the right way to invoke a particular JavaScript function while being mindful of state. These situations are common, and they consume significant time and cognitive resources. They are also what I call “robot work,” as in the kind of repetitive rote work you can imagine a robot doing. 

Now imagine having a skilled co-worker on call, at all times, ready to help you find and explain the right formula, document that inscrutable script, and refactor or even build from scratch that JavaScript function you need.

That’s what AI can be for us: Just-In-Time assistants for all the tedious, time consuming, and rote robot work taking up our valuable time and cognitive capacity.

If you’re a developer, you can experience this future today via various AI integrations including GitHub Copilot and ChatGPT.

GitHub Copilot coupled with the new GitHub Copilot Labs extension in VS Code gives you a pair programming assistant right in your development environment. Highlight any block of code and in the Copilot Labs panel you can ask for an explanation of the code, have it translated into another (applicable) code language, use a series of “brushes” on it including making the code more readable, adding types, cleaning, chunking, even documenting. You can even use Copilot to write and run tests on your code.

A myriad of ChatGPT extensions including Ali Gençay’s ChatGPT for VS Code does much the same, via a slightly different route. Authenticate the extension with OpenAI’s ChatGPT API, highlight any code, and you can ask ChatGPT to add tests, find bugs, optimize, explain, and add comments automatically. You also get the ability to start a full chat with ChatGPT in a dedicated panel right inside the editor where you can talk to the AI in more detail about whatever you want. 

Time and Energy

This past week I’ve been working on a large project involving code written by someone else. The JavaScript of the project has been co-located in two giant files, and is an unruly mix of carbon copies of standard components and functions, modified code copied from documentation sites, and custom code. As is often the case, the documentation is lacking where it’s most needed.

For my project, I need to refactor (rewrite to do the same code in a different way) large parts of the code, and add extensive documentation along the way. That means figuring out how the code works, adding comments to clearly document and explain it, reorganizing the code so it can be split into components, cleaning up and updating the code to modern standards, and finally splitting it up into discrete modules. This is robot work: rote repetitive tasks consuming significant time and cognitive energy better spent focusing on what the code is for: Serving as an example for a course on JavaScript build processes.

In the past, I’d end up investing several days doing this work. Now, with the help of AI, the workload is significantly reduced. This, I think, is an instructive example of how our future with AI can unfold: with AI assisting us as we do our work rather than take over that work.

Refactor this

Both GitHub Copilot and ChatGPT are conversational AIs. You chat with them as you would chat to a person. You can ask questions, give instructions, and ask them to perform tasks for you. Using AIs as pair programmers you do a combination of all of this and more.

If you’re using the VS Code extensions mentioned above, they are already set up for the correct context. In the case of ChatGPT you can also use it as a stand-alone pair-programmer with some basic setup:

To start, set the stage (literally) by instructing the AI on how you want it to behave. In a new chat, provide a prompt similar to this:

“You are a pair programmer specializing in JavaScript. You help refactor code to bring it up to ES6 standards. Is that understood?”

The format here is “you are a [some role]. You [perform some skill/action]. Is that understood?” The last question gives the AI an opportunity to state how it is setting itself up based on your instructions and gives you an opportunity to provide further instructions. In my case ChatGPT responded as follows:

“Yes, that is understood. As a language model, I am able to understand and provide assistance with various programming languages, including JavaScript and its latest version, ES6. I can help refactor code to bring it up to current standards, as well as provide guidance on best practices for writing maintainable, efficient code.”

From here you can for example paste in a JavaScript function and ask the AI to help find an error and resolve it:

In response, ChatGPT provides an explanation of the errors it discovered, prototype examples of solutions to the issues, and finally a full refactoring of the pasted code with the issues resolved.

This kind of contextual response not only helps solve immediate problems, but also teaches you what’s wrong and how to fix it.

This is invaluable for people learning to code and people working with code in any capacity which is why I’d strongly discourage any teacher or manager who is right now trying to figure out how to block people from using AIs in their work. AIs reduce the need for Googling or looking up code examples on documentation sites, coding forums, and open source repositories. Instead they give you contextual explanations and references related to your specific code, and even help you with refactoring. This is the future of work, and gives us more capabilities as workers.

• Have some code you can’t make heads or tails of? AI can explain what it does. Computers are much better at parsing logic based languages than humans, and conversational AI like ChatGPT are specifically constructed to output human-sounding language making them ideal tools for decrypting complex code for human consumption.
• Have some code in need of documentation? AI can write a function description, inline comments, or whatever you prefer based on your instructions.
• Need to refactor based on specific parameters? AI can get you started.
• I could go on but I think you get the idea.

I’ve worked alongside these AI pair programmers for the past year and a bit, and I can say with absolute conviction these tools and materials will make our lives better if we use them right and integrate them in our lives as helpers for rather than replacements of human labor.

In my experience, pair programming with an AI feels like working with an overly polite person with encyclopedic knowledge of coding and no awareness of what they don’t know. And this constitutes just our first timid steps into the infinite possibility space we are entering as AIs become our assistants.

The beginning of the beginning

As you interact with AI today, be constantly aware of where you are: At the beginning of the beginning of a new era. While these tools are powerful, they are not omnipotent. Far from it. They are shallow, error prone, and while they sound convincing they cannot be trusted. A good mental model for what they produce right now is bullshit as defined by Harry G. Frankfurt: It looks true, and it may be true, but some of the time it’s just plain wrong and the AI will still present it as the truth. While they talk like humans, AIs are not conscious or sentient or aware. They have no human comprehension of your question or their answer. They are advanced pattern recognition systems who tumble down enormously complex decision trees any time a prompt is provided to issue human-sounding strings of text (or code) with a statistically high probability of being the kind of answer that is considered correct by their human trainers. 

When I asked ChatGPT to correct a function containing a deprecated method, it corrected the syntax of the function but kept the deprecated method. When I told it the method was deprecated, it omitted it and refactored the code, but the result used a similar-sounding method that serves a very different purpose and was therefore non-functional and just plain wrong.

When I asked ChatGPT to find an error in a large body of code, it found two real errors and invented a third one, going as far as referencing use of a method that wasn’t even in the code provided.

These examples highlight why I see AIs as tools and materials rather than replacements for human labor. They have no understanding, no contextual awareness, no ability to do creative or lateral thinking. A human still needs to be in the loop; to make sure the output meets parameters, does what was intended, and follows best practices and standards (not to mention hold ethical responsibility for the work created). These things we call “AI” are very much artificial, but they are not intelligent. 

Intelligence is added by the human operator.

Even so, the pair programming offered by these prototype AIs is an enormous leap forward for human workers. And you can easily see how this type of AI-driven assistance can be extended to other work and other tasks. 

I’ve come to think of them as overconfident colleagues with a lack of self-awareness. Because of how they are “trained” – being fed large volumes of data from a corpus lifted off the internet – their “knowledge” is limited to the coding world of two years ago. Therefore, when it comes to modern features, frameworks, techniques, and standards released in the past two years, our current AIs know naught, and more importantly do not know what they do not know. Therefore, if you’re writing code on the bleeding edge of standards, you’re on your own. Or better yet: You’re training your future AI pair programmer! So the pressure is on to get it right!

The future is today

Having seen what AIs can do today, I desperately wish I had a looking glass to see what the future of work looks like. The potential here is infinite. The very best AI tools we have today are prototypes and MVPs trained on old data and limited in their scope. The AIs we’ll have a year from now, five years from now, ten years from now will be beyond anything we can imagine. And with these tools and materials in hand we can choose to build meaningful futures for everyone where we all have the capabilities to be and do what we have reason to value.

The future we are stepping into today is a future where AI is part of our lives, our work, our communities and our society. If you are alive today, and especially if you find yourself in a job, you are in the right place at the right time: These next few years is when we collectively get to figure out how AI fits into our lives and our work. This is when we set the stage for our futures with AI, and we all have a part to play. The work starts by asking yourself in what parts of your life you act like a robot, and whether you’re willing to part with that work and let an AI do it for you so you can do something else. 

If we do this right, AI will allow us to reclaim our time to be human.

—

Cross-posted to LinkedIn.

“Language shapes the way we think, and determines what we can think about.”

Benjamin Lee Whorf

Before we begin, I asked ChatGPT to rewrite this article at a 4th grade reading level. You can read the result here.

Artificial? Yes. Intelligent? Not even close. It is not without reason things like ChatGPT are called “AI” or “Artificial Intelligence.” We humans have a propensity for anthropomorphizing – attribute human characteristics to – things that are not human. Thus if we are told something is intelligent, let’s say a very large computer system we can submit questions and get answers from, we look for intelligence in that thing. And if that thing is trained on our own language and art and mathematics and code, it will appear to us as intelligent because its training materials came from intelligent beings: Us ourselves.

“Artificial Intelligence” is a clever marketing term for computer models designed to appear intelligent even though they are not.

So, as we crash headfirst into the AI present and future, we need to reset our mental model before we start believing these things we call “Artificial Intelligences” are actually intelligent (again, they are not).

Tools and Materials

I propose we all start thinking of these things we call “AI” as tools and materials. Because that’s what they are and that’s how we’ll end up using them.

Sometimes we’ll use them as tools the same way we use our phones and computers and the apps on them as tools. Sometimes we’ll use them and what they produce as materials the same way we use printed fabrics and code snippets to create things. And sometimes we’ll use them as both tools and materials the same way we use word processing applications first as a tool with which we write a body of text and then a material as the thesaurus function helps us use more fanciful words and phrases.

Here are some basic examples to help you build the mental model:

AI as a tool performs a task for us:

• Fill out tax forms, write contracts and legal documents.
• Summarize text, rewrite text to a specific reading level.
• Write code.
• Online shopping including booking flights and hotels etc.
• Any interaction with any CSR.
• Magic eraser for images, video, and audio.

AI as a material generates something for us:

• Simple stories.
• Plot lines for stories.
• News articles and summaries.
• Images and other art.
• Variants of a layout, or a theme, or an image, or a painting.

Thinking of AI as tools and materials rather than intelligent things with magical human-like powers is an essential mental shift as we figure out how to fit these things into our lives and our world. We have to move away from the linguistic trick their creators foisted upon us with their naming, and move towards the practical realities of what these things really are:

AI are if-this-then-that machines using enormously complex decision trees generated by ingesting all available writings, imagery, and other human-made materials and filtering that data through pattern-matching algorithms.

They are regurgitation machines echoing our own works back to us.

And just like we are drawn to our own image every time we pass a mirrored surface, we are drawn to the echoes of ourselves in the output of these machines.

Shallow Work and Human Creativity

Asked for one word to describe AIs, my immediate answer is “shallow.” You’ve probably felt this yourself without being able to put your finger on it. Let me explain:

There is a bland uniformity to AI output. It’s easiest to notice in generative AI images. Once you’ve been exposed to enough of them, they start taking on a very specific “AI-ness.” For all their variety, there is something recognizable about them – some defining feature that sets them apart from what we recognize at human-made images. That thing is shallowness.

AIs are conservative in the sense they conserve and repeat what already exists. They don’t come up with anything new. They are also elitist in the sense they lean towards what is predominant, what there is more of. They are swayed by trends and popularity and amplify whatever majority opinion they find in their training data.

This makes their output bland and uniform and shallow like a drunk first-year philosophy student at a bar: The initial conversation may be interesting, but after a few minutes you notice there’s little substance behind the bravado. I’ve been that drunk first-year philosophy student so I know what I’m talking about.

This means while AIs are great at doing shallow rote work, they have no ability to bring anything new to the table. They lack creativity and ingenuity and lateral thinking skills because these skills require intelligence. And AIs are not intelligent; they just play intelligent on TV.

Will an AI take my job?

Our instinctual response any new technology is “will it take my job?” It’s a valid question: Jobs are essential for us to be able to make a living in this free-market capitalist delusion we call “modern society,” yet job creators have a tendency to let go of expensive human workers if they can replace them with less expensive alternatives like self-checkout kiosks that constantly need to be reset by a staff member because you put the banana in the bagging area before you chose whether to donate $2 to a children’s charity, or automated “voice assistants” that never have the answers to your customer service questions and only pass you to an actual human once you’ve repeated the correct incantation of profanity (try it, it totally works!)

So now that we have these things some clever marketing people have told us to call “AI,” are they coming for your job? Well, that depends:

If your job is shallow and constitutes mainly rote work, there’s a good chance an AI will enter your life very soon – as in within months – and become part of the toolkit you use to get your job done quicker. And if it turns out that AI can be trained to do your job without your intervention (by having you use it and thereby training it), there’s a non-zero chance it will eventually replace you. That chance hinges more on corporate greed than it does AI ability though.

If your job involves any type of creative, or deep, or lateral, or organizational, or original, or challenging, or novel thinking, AI will not take your job because AI can’t do any of those things. You’ll still work with AI – probably within months – and the AI may alleviate you of a lot of the rote work you are currently doing that takes your attention away from what you were actually hired to do – but the AI is unlikely to replace you. Unless corporate greed gets in the way. Which it often does because of the aforementioned free-market capitalist delusion we call “modern society.”

What we all have to come to terms with today is we’re long past the point of no return when it comes to AI. While technology is not inevitable, technology often becomes so entrenched it is impossible to … un-entrench it. That’s where we are with AI. No matter where you live and what you do for work, for school, or in your own time, you’re already interacting with AIs in more ways that you can imagine. And these AIs are going to become part of your work, your school, and your home life whether you want them or not.

Our job now is to talk to one another about what role these things called “AI” are going to play in our lives. How do we use them in ways that don’t take jobs away from the humans who need them the most – the historically marginalized and excluded people who tend to hold jobs comprising mainly shallow rote work? How do we build them in ways that don’t cannibalize the creative works of artists and writers and coders and teachers? How do we incorporate AI into education to improve learning outcomes for students and build a more informed and skilled populace? How do we wrench control over our AI future from the surveillance capitalists and longtermists currently building the world to their libertarian techno-utopian visions?

How do we use AI and all technology to create human flourishing and build futures in which we all have the capabilities to be and do what we have reason to value?

If we don’t talk about the future, the future becomes something that happens to us. Let’s have this conversation.

Cross-posted to LinkedIn.