Categories
Uncategorized

We cool?

The democratic caucus fiasco in Iowa is an example of an important thing:

Civic orgs, like the DNC or the government, need to stop using private contracting companies to build their closed source software. Democracy is at stake.


First let me establish some definitions, if you already know this stuff you can skip it, but I just want to make sure this article is accessible. Definitions have also been simplified for the purposes of addressing context.

What does “closed source software mean”?

“Source” refers to the code that a programmer writes to make a computer program. Source code is then generally compiled down into a file that the machine can read to execute the program. This is called “binary code” or you might refer to it as a native app or an executable.

Closed source means that you get the compiled code, but you don’t get the original code that the programmer wrote. This is how most consumers use software, as compiled code. Examples are, the operating system Microsoft Windows 10, apps on the Apple app store, or games on your PC or in a console.

Open source code is code which you have access to the original code. This means you can edit and change your program. You can copy it. You can reuse it for your own purposes.


Open source is necessary for transparent democratic governance.

Let’s start by just agreeing to assume that transparency is a necessary aspect of a democracy. A democracy whose inner workings are inaccessible to the voter might as well be a black box. There’s no way to really hold it accountable or know it’s working.

Open source in civic orgs is the only way to ultimately insure transparency in government. We have reports that candidate Pete Buttigieg paid money (about 40k) to the company Shadow, funded by superPac ACRONYM (fire the show writer please). Now we don’t have to immediately assume this is corruption, this could be innocent. But it’s kind of hard without being able to see the code ourselves and audit things.

There are ways to do this with open source projects. First the source is directly available so people can just see it. Not only that but you can verify your vote!

The open source blockchain project Hyperledger was used to make the West Virginian experiment in smartphone voting, Voatz.

Helios provides a voting server system that allows voting by email, that uses hashes to confirm your vote and keep your vote anonymous at the same time. Helios is obviously not a good pick, design wise, for national elections but it’s encryption and verification method certainly are.

If transparency is so important to democracy then why isn’t the infrastructure being developed for democracy also being transparently developed in the most transparent way possible?

Open source is extremely common and yet most civic org code is closed source. Why?

It’s simple.

Money

1.

Closed source companies sold fearful portents about the security of open source in order to secure contracts with civic organizations.
Security is not an issue with open source. In fact it is generally speaking more secure than closed source software. The vast majority of online servers are Linux based. Microsoft itself is one of the biggest contributors to Linux and now runs many of their servers on Linux. The preferred OS for hacker and crypto types is a Linux variant called Kali.

The reason why is simple. A team of 3 or even 300 devs are nothing compared to what the eyeballs of 30,000 can spot. Bugs will happen. But like a body with a healthy immune system, the only way to deal with potential problems is exposure and adaption. Open source projects can’t hide behind obfuscation, which is more often a crutch than anything else. Why were we told anything different? As I said, it’s about money. But this isn’t the only way it’s about money.

2.

Leaders at civic orgs have been incentivized to buy from private contractors. This is a problem for a number of reason:

2.1

It strips internal departments of their tribal knowledge, and reduces the investment in improving and training staff.

Let’s say a contractor came in and made a web app. You might do this because it’s more cost effective in the short term to bring in an expensive but short term team, that already knows how to make the web app. They just come in and make the app. Then some of them might stay awhile to train up some internal staff for hand off, but then from then on the web app is largely just being maintained. This is essentially a form of deskilling.

This is a problem because there’s only so much you can document and teach. A lot of experience is gained through the process of building something yourself. And employees are not static, they have growth potential. Dividing out efforts between internal and external teams means losing a substantial part of the people growth and it disincentivizes investing further growth in internal employees because they largely are relegated to a role of keeping the ship on course.

2.2

Private external contracting companies do not have an incentive to open source their work, generally speaking. They are a private company after all.

2.3

Between these first two you get this third issue. Self dealing behavior.

That Pete Buttigieg payment to Shadow? Well it might interest you to learn that the Tara McGowan, the owner of ACRONYM, has a husband named Michael Halle, and he’s a senior strategist for Pete…

We can’t prove anything happened here obviously but the power to award contracts has long been used as a tool of “machine politics” and graft. There is value created in the process of building things (the cultivation of tribal knowledge and the development of libraries and systems to reuse) and that value is externalized from the public space and internalized to the private space. Make profits private, make losses public. The old Bank Bailout strategy.

2. 4

Contracts are a way for politicians to benefit those they like and people who will support them. Often rich people. Often superPacs. Lawerence Lessig makes the argument for how superPacs corrupt the system in his TED talk here. The same basic principle applies.


Open Source solves problems you didn’t even know you had

Besides increasing transparency/security there are additional bonuses to shifting our government from private closed source projects to public open source projects.

Code Reuse:

Sure, growing an internal team long term is going to be more expensive upfront but it’s an investment with a very very long tail.

A project which is used in Iowa can be taken and repurposed in another caucus state. Or can be repurposed by a caucus like structure in another country! All programming and computer science is like this. It’s people building on the work who built on the work of other people. The initial investment in ARPAnet, and research into computer processing was all government funded, as a big upfront investment that has had a long tail of returns. We need to reinvest in our technological infrastructure, not just so that crap like Iowa doesn’t happen but so that there is a rising tide to lift all boats. Economists Mariana Mazzucato makes this argument in her book The Entrepreneurial State.

Bottom up, grassroots experiments in democracy:

There are lots of experiments that can be attempted at local levels with the possible overflow into larger levels, bottom up style, rather than top down.

Remember how I mentioned Helios and Voatz earlier? Helios allows for mathematically certain verification that your vote was counted. But it also allows for alternatives to First Past the Post (the traditional voting you know of), such as Ranked Choice voting. This is simply the tip of the iceberg.

I recommend checking out these two other projects, purely from a design perspective, because they show that there is a lot of of further changes to the system that could be made to improve it.

Kialois an argument and discussion aid tool that structures conversations in ways that are meant to create proper citation, avoid logical fallacies, and map the difference of opinion across many levels of an issue.

Loomiois an open source “decision making platform”. Think of it like slack but for getting stuff decided instead of endlessly flooded with gifs. It operates on principles of discussion/debate that were developed by organizers of Occupy Wallstreet.

Provide a method to improve democracy:

Think of democracy as a code project. If you’re reading this article you might not know much about code, but you can observe these basic points I’m going to mention in a lot of human systems around you, music scenes, the “vibe” of a relationship, the evolution of language and how we pass on knowledge in communities.

Code projects, particularly open source ones, usually start with someone trying to solve a problem. They go at solving that problem, and they write this initial program. That program is going to have certain decisions they made for certain reasons that at the time made sense but maybe don’t now. Or perhaps there were bugs in the code, so the developers patched it up with duct tape and chewing gum, which works but if you push the wrong lever or button the whole thing just shivers and shakes and falls apart in a heap on the floor.

In tech circles this is referred to as “tech debt”. This is obviously a problem. It’s a problem not just with programming code, it’s a problem with our system of governance in general. Think of laws as similar to programming code. Did you know that there’s nowhere in the constitution where it is stated that the job of the Supreme Court is to determine constitutionality?

That’s because the Founders didn’t specify that…why? I have no clue. But the constitution is the first of it’s kind in modern history, an explicitly democratic republic. It’s not surprising that maybe they didn’t get it perfect the first time. I mean, they did immediately amend it, after all. It’s a document which was built to evolve.

But the problem is that it’s the beta. Or as devs say “devving in prod”. There’s been a lot of innovations in democracy in the last 250 years. Alternative choice! Parliamentary systems! Mathematical models to prevent Gerrymandering!

But what we have instead is the beta. And it’s old and it’s crusty and the chewing gum is starting to get brittle. We need real solutions.

A big part of fixing this is fixing our infrastructure. And open source is key to that.


One last thing:

This won’t be easy.

Open source is not known for being user friendly. Neither is politics.

Those who make open source are also the ones who use it so it tends to trend more towards technical users. This is much less true than it used to be. But it still is a little true. But this is not necessarily a bad thing.

Technology should not be a black box. Just as civics class is a necessary part of being a citizen, technical knowledge is a necessary part of being a digital citizen. This isn’t going to be easy but… like really when has democracy ever been “easy”, right?

Democracy can’t happen without you, as User, and Author. Do no let an elite political class define the system you use. Government for The People, by The People, Author and User one and the same.

mr. barnie sanders saying hello
Categories
Uncategorized

Man I wish I’d learned python instead

The democratic caucus fiasco in Iowa is an example of an important thing:

Civic orgs, like the DNC or the government, need to stop using private contracting companies to build their closed source software. Democracy is at stake.


First let me establish some definitions, if you already know this stuff you can skip it, but I just want to make sure this article is accessible. Definitions have also been simplified for the purposes of addressing context.

What does “closed source software mean”?

“Source” refers to the code that a programmer writes to make a computer program. Source code is then generally compiled down into a file that the machine can read to execute the program. This is called “binary code” or you might refer to it as a native app or an executable.

Closed source means that you get the compiled code, but you don’t get the original code that the programmer wrote. This is how most consumers use software, as compiled code. Examples are, the operating system Microsoft Windows 10, apps on the Apple app store, or games on your PC or in a console.

Open source code is code which you have access to the original code. This means you can edit and change your program. You can copy it. You can reuse it for your own purposes.


Open source is necessary for transparent democratic governance.

Let’s start by just agreeing to assume that transparency is a necessary aspect of a democracy. A democracy whose inner workings are inaccessible to the voter might as well be a black box. There’s no way to really hold it accountable or know it’s working.

Open source in civic orgs is the only way to ultimately insure transparency in government. We have reports that candidate Pete Buttigieg paid money (about 40k) to the company Shadow, funded by superPac ACRONYM (fire the show writer please). Now we don’t have to immediately assume this is corruption, this could be innocent. But it’s kind of hard without being able to see the code ourselves and audit things.

There are ways to do this with open source projects. First the source is directly available so people can just see it. Not only that but you can verify your vote!

The open source blockchain project Hyperledger was used to make the West Virginian experiment in smartphone voting, Voatz.

Helios provides a voting server system that allows voting by email, that uses hashes to confirm your vote and keep your vote anonymous at the same time. Helios is obviously not a good pick, design wise, for national elections but it’s encryption and verification method certainly are.

If transparency is so important to democracy then why isn’t the infrastructure being developed for democracy also being transparently developed in the most transparent way possible?

Open source is extremely common and yet most civic org code is closed source. Why?

It’s simple.

Money

1.

Closed source companies sold fearful portents about the security of open source in order to secure contracts with civic organizations.
Security is not an issue with open source. In fact it is generally speaking more secure than closed source software. The vast majority of online servers are Linux based. Microsoft itself is one of the biggest contributors to Linux and now runs many of their servers on Linux. The preferred OS for hacker and crypto types is a Linux variant called Kali.

The reason why is simple. A team of 3 or even 300 devs are nothing compared to what the eyeballs of 30,000 can spot. Bugs will happen. But like a body with a healthy immune system, the only way to deal with potential problems is exposure and adaption. Open source projects can’t hide behind obfuscation, which is more often a crutch than anything else. Why were we told anything different? As I said, it’s about money. But this isn’t the only way it’s about money.

2.

Leaders at civic orgs have been incentivized to buy from private contractors. This is a problem for a number of reason:

2.1

It strips internal departments of their tribal knowledge, and reduces the investment in improving and training staff.

Let’s say a contractor came in and made a web app. You might do this because it’s more cost effective in the short term to bring in an expensive but short term team, that already knows how to make the web app. They just come in and make the app. Then some of them might stay awhile to train up some internal staff for hand off, but then from then on the web app is largely just being maintained. This is essentially a form of deskilling.

This is a problem because there’s only so much you can document and teach. A lot of experience is gained through the process of building something yourself. And employees are not static, they have growth potential. Dividing out efforts between internal and external teams means losing a substantial part of the people growth and it disincentivizes investing further growth in internal employees because they largely are relegated to a role of keeping the ship on course.

2.2

Private external contracting companies do not have an incentive to open source their work, generally speaking. They are a private company after all.

2.3

Between these first two you get this third issue. Self dealing behavior.

That Pete Buttigieg payment to Shadow? Well it might interest you to learn that the Tara McGowan, the owner of ACRONYM, has a husband named Michael Halle, and he’s a senior strategist for Pete…

We can’t prove anything happened here obviously but the power to award contracts has long been used as a tool of “machine politics” and graft. There is value created in the process of building things (the cultivation of tribal knowledge and the development of libraries and systems to reuse) and that value is externalized from the public space and internalized to the private space. Make profits private, make losses public. The old Bank Bailout strategy.

2. 4

Contracts are a way for politicians to benefit those they like and people who will support them. Often rich people. Often superPacs. Lawerence Lessig makes the argument for how superPacs corrupt the system in his TED talk here. The same basic principle applies.


Open Source solves problems you didn’t even know you had

Besides increasing transparency/security there are additional bonuses to shifting our government from private closed source projects to public open source projects.

Code Reuse:

Sure, growing an internal team long term is going to be more expensive upfront but it’s an investment with a very very long tail.

A project which is used in Iowa can be taken and repurposed in another caucus state. Or can be repurposed by a caucus like structure in another country! All programming and computer science is like this. It’s people building on the work who built on the work of other people. The initial investment in ARPAnet, and research into computer processing was all government funded, as a big upfront investment that has had a long tail of returns. We need to reinvest in our technological infrastructure, not just so that crap like Iowa doesn’t happen but so that there is a rising tide to lift all boats. Economists Mariana Mazzucato makes this argument in her book The Entrepreneurial State.

Bottom up, grassroots experiments in democracy:

There are lots of experiments that can be attempted at local levels with the possible overflow into larger levels, bottom up style, rather than top down.

Remember how I mentioned Helios and Voatz earlier? Helios allows for mathematically certain verification that your vote was counted. But it also allows for alternatives to First Past the Post (the traditional voting you know of), such as Ranked Choice voting. This is simply the tip of the iceberg.

I recommend checking out these two other projects, purely from a design perspective, because they show that there is a lot of of further changes to the system that could be made to improve it.

Kialois an argument and discussion aid tool that structures conversations in ways that are meant to create proper citation, avoid logical fallacies, and map the difference of opinion across many levels of an issue.

Loomiois an open source “decision making platform”. Think of it like slack but for getting stuff decided instead of endlessly flooded with gifs. It operates on principles of discussion/debate that were developed by organizers of Occupy Wallstreet.

Provide a method to improve democracy:

Think of democracy as a code project. If you’re reading this article you might not know much about code, but you can observe these basic points I’m going to mention in a lot of human systems around you, music scenes, the “vibe” of a relationship, the evolution of language and how we pass on knowledge in communities.

Code projects, particularly open source ones, usually start with someone trying to solve a problem. They go at solving that problem, and they write this initial program. That program is going to have certain decisions they made for certain reasons that at the time made sense but maybe don’t now. Or perhaps there were bugs in the code, so the developers patched it up with duct tape and chewing gum, which works but if you push the wrong lever or button the whole thing just shivers and shakes and falls apart in a heap on the floor.

In tech circles this is referred to as “tech debt”. This is obviously a problem. It’s a problem not just with programming code, it’s a problem with our system of governance in general. Think of laws as similar to programming code. Did you know that there’s nowhere in the constitution where it is stated that the job of the Supreme Court is to determine constitutionality?

That’s because the Founders didn’t specify that…why? I have no clue. But the constitution is the first of it’s kind in modern history, an explicitly democratic republic. It’s not surprising that maybe they didn’t get it perfect the first time. I mean, they did immediately amend it, after all. It’s a document which was built to evolve.

But the problem is that it’s the beta. Or as devs say “devving in prod”. There’s been a lot of innovations in democracy in the last 250 years. Alternative choice! Parliamentary systems! Mathematical models to prevent Gerrymandering!

But what we have instead is the beta. And it’s old and it’s crusty and the chewing gum is starting to get brittle. We need real solutions.

A big part of fixing this is fixing our infrastructure. And open source is key to that.


One last thing:

This won’t be easy.

Open source is not known for being user friendly. Neither is politics.

Those who make open source are also the ones who use it so it tends to trend more towards technical users. This is much less true than it used to be. But it still is a little true. But this is not necessarily a bad thing.

Technology should not be a black box. Just as civics class is a necessary part of being a citizen, technical knowledge is a necessary part of being a digital citizen. This isn’t going to be easy but… like really when has democracy ever been “easy”, right?

Democracy can’t happen without you, as User, and Author. Do no let an elite political class define the system you use. Government for The People, by The People, Author and User one and the same.

mr. barnie sanders saying hello
Categories
Uncategorized

Another Title Here

The democratic caucus fiasco in Iowa is an example of an important thing:

Civic orgs, like the DNC or the government, need to stop using private contracting companies to build their closed source software. Democracy is at stake.


First let me establish some definitions, if you already know this stuff you can skip it, but I just want to make sure this article is accessible. Definitions have also been simplified for the purposes of addressing context.

What does “closed source software mean”?

“Source” refers to the code that a programmer writes to make a computer program. Source code is then generally compiled down into a file that the machine can read to execute the program. This is called “binary code” or you might refer to it as a native app or an executable.

Closed source means that you get the compiled code, but you don’t get the original code that the programmer wrote. This is how most consumers use software, as compiled code. Examples are, the operating system Microsoft Windows 10, apps on the Apple app store, or games on your PC or in a console.

Open source code is code which you have access to the original code. This means you can edit and change your program. You can copy it. You can reuse it for your own purposes.


Open source is necessary for transparent democratic governance.

Let’s start by just agreeing to assume that transparency is a necessary aspect of a democracy. A democracy whose inner workings are inaccessible to the voter might as well be a black box. There’s no way to really hold it accountable or know it’s working.

Open source in civic orgs is the only way to ultimately insure transparency in government. We have reports that candidate Pete Buttigieg paid money (about 40k) to the company Shadow, funded by superPac ACRONYM (fire the show writer please). Now we don’t have to immediately assume this is corruption, this could be innocent. But it’s kind of hard without being able to see the code ourselves and audit things.

There are ways to do this with open source projects. First the source is directly available so people can just see it. Not only that but you can verify your vote!

The open source blockchain project Hyperledger was used to make the West Virginian experiment in smartphone voting, Voatz.

Helios provides a voting server system that allows voting by email, that uses hashes to confirm your vote and keep your vote anonymous at the same time. Helios is obviously not a good pick, design wise, for national elections but it’s encryption and verification method certainly are.

If transparency is so important to democracy then why isn’t the infrastructure being developed for democracy also being transparently developed in the most transparent way possible?

Open source is extremely common and yet most civic org code is closed source. Why?

It’s simple.

Money

1.

Closed source companies sold fearful portents about the security of open source in order to secure contracts with civic organizations.
Security is not an issue with open source. In fact it is generally speaking more secure than closed source software. The vast majority of online servers are Linux based. Microsoft itself is one of the biggest contributors to Linux and now runs many of their servers on Linux. The preferred OS for hacker and crypto types is a Linux variant called Kali.

The reason why is simple. A team of 3 or even 300 devs are nothing compared to what the eyeballs of 30,000 can spot. Bugs will happen. But like a body with a healthy immune system, the only way to deal with potential problems is exposure and adaption. Open source projects can’t hide behind obfuscation, which is more often a crutch than anything else. Why were we told anything different? As I said, it’s about money. But this isn’t the only way it’s about money.

2.

Leaders at civic orgs have been incentivized to buy from private contractors. This is a problem for a number of reason:

2.1

It strips internal departments of their tribal knowledge, and reduces the investment in improving and training staff.

Let’s say a contractor came in and made a web app. You might do this because it’s more cost effective in the short term to bring in an expensive but short term team, that already knows how to make the web app. They just come in and make the app. Then some of them might stay awhile to train up some internal staff for hand off, but then from then on the web app is largely just being maintained. This is essentially a form of deskilling.

This is a problem because there’s only so much you can document and teach. A lot of experience is gained through the process of building something yourself. And employees are not static, they have growth potential. Dividing out efforts between internal and external teams means losing a substantial part of the people growth and it disincentivizes investing further growth in internal employees because they largely are relegated to a role of keeping the ship on course.

2.2

Private external contracting companies do not have an incentive to open source their work, generally speaking. They are a private company after all.

2.3

Between these first two you get this third issue. Self dealing behavior.

That Pete Buttigieg payment to Shadow? Well it might interest you to learn that the Tara McGowan, the owner of ACRONYM, has a husband named Michael Halle, and he’s a senior strategist for Pete…

We can’t prove anything happened here obviously but the power to award contracts has long been used as a tool of “machine politics” and graft. There is value created in the process of building things (the cultivation of tribal knowledge and the development of libraries and systems to reuse) and that value is externalized from the public space and internalized to the private space. Make profits private, make losses public. The old Bank Bailout strategy.

2. 4

Contracts are a way for politicians to benefit those they like and people who will support them. Often rich people. Often superPacs. Lawerence Lessig makes the argument for how superPacs corrupt the system in his TED talk here. The same basic principle applies.


Open Source solves problems you didn’t even know you had

Besides increasing transparency/security there are additional bonuses to shifting our government from private closed source projects to public open source projects.

Code Reuse:

Sure, growing an internal team long term is going to be more expensive upfront but it’s an investment with a very very long tail.

A project which is used in Iowa can be taken and repurposed in another caucus state. Or can be repurposed by a caucus like structure in another country! All programming and computer science is like this. It’s people building on the work who built on the work of other people. The initial investment in ARPAnet, and research into computer processing was all government funded, as a big upfront investment that has had a long tail of returns. We need to reinvest in our technological infrastructure, not just so that crap like Iowa doesn’t happen but so that there is a rising tide to lift all boats. Economists Mariana Mazzucato makes this argument in her book The Entrepreneurial State.

Bottom up, grassroots experiments in democracy:

There are lots of experiments that can be attempted at local levels with the possible overflow into larger levels, bottom up style, rather than top down.

Remember how I mentioned Helios and Voatz earlier? Helios allows for mathematically certain verification that your vote was counted. But it also allows for alternatives to First Past the Post (the traditional voting you know of), such as Ranked Choice voting. This is simply the tip of the iceberg.

I recommend checking out these two other projects, purely from a design perspective, because they show that there is a lot of of further changes to the system that could be made to improve it.

Kialois an argument and discussion aid tool that structures conversations in ways that are meant to create proper citation, avoid logical fallacies, and map the difference of opinion across many levels of an issue.

Loomiois an open source “decision making platform”. Think of it like slack but for getting stuff decided instead of endlessly flooded with gifs. It operates on principles of discussion/debate that were developed by organizers of Occupy Wallstreet.

Provide a method to improve democracy:

Think of democracy as a code project. If you’re reading this article you might not know much about code, but you can observe these basic points I’m going to mention in a lot of human systems around you, music scenes, the “vibe” of a relationship, the evolution of language and how we pass on knowledge in communities.

Code projects, particularly open source ones, usually start with someone trying to solve a problem. They go at solving that problem, and they write this initial program. That program is going to have certain decisions they made for certain reasons that at the time made sense but maybe don’t now. Or perhaps there were bugs in the code, so the developers patched it up with duct tape and chewing gum, which works but if you push the wrong lever or button the whole thing just shivers and shakes and falls apart in a heap on the floor.

In tech circles this is referred to as “tech debt”. This is obviously a problem. It’s a problem not just with programming code, it’s a problem with our system of governance in general. Think of laws as similar to programming code. Did you know that there’s nowhere in the constitution where it is stated that the job of the Supreme Court is to determine constitutionality?

That’s because the Founders didn’t specify that…why? I have no clue. But the constitution is the first of it’s kind in modern history, an explicitly democratic republic. It’s not surprising that maybe they didn’t get it perfect the first time. I mean, they did immediately amend it, after all. It’s a document which was built to evolve.

But the problem is that it’s the beta. Or as devs say “devving in prod”. There’s been a lot of innovations in democracy in the last 250 years. Alternative choice! Parliamentary systems! Mathematical models to prevent Gerrymandering!

But what we have instead is the beta. And it’s old and it’s crusty and the chewing gum is starting to get brittle. We need real solutions.

A big part of fixing this is fixing our infrastructure. And open source is key to that.


One last thing:

This won’t be easy.

Open source is not known for being user friendly. Neither is politics.

Those who make open source are also the ones who use it so it tends to trend more towards technical users. This is much less true than it used to be. But it still is a little true. But this is not necessarily a bad thing.

Technology should not be a black box. Just as civics class is a necessary part of being a citizen, technical knowledge is a necessary part of being a digital citizen. This isn’t going to be easy but… like really when has democracy ever been “easy”, right?

Democracy can’t happen without you, as User, and Author. Do no let an elite political class define the system you use. Government for The People, by The People, Author and User one and the same.

mr. barnie sanders saying hello
Categories
Uncategorized

Short Title

Testing a short title hereTesting a short title hereTesting a short title hereTesting a short title hereTesting a short title hereTesting a short title hereTesting a short title hereTesting a short title hereTesting a short title hereTesting a short title hereTesting a short title hereTesting a short title hereTesting a short title hereTesting a short title hereTesting a short title hereTesting a short title hereTesting a short title hereTesting a short title hereTesting a short title hereTesting a short title hereTesting a short title hereTesting a short title hereTesting a short title hereTesting a short title hereTesting a short title hereTesting a short title hereTesting a short title hereTesting a short title hereTesting a short title hereTesting a short title hereTesting a short title hereTesting a short title hereTesting a short title hereTesting a short title hereTesting a short title here

Testing a short title hereTesting a short title hereTesting a short title hereTesting a short title hereTesting a short title hereTesting a short title hereTesting a short title hereTesting a short title hereTesting a short title hereTesting a short title hereTesting a short title hereTesting a short title hereTesting a short title hereTesting a short title hereTesting a short title hereTesting a short title hereTesting a short title hereTesting a short title here

Testing a short title hereTesting a short title hereTesting a short title hereTesting a short title hereTesting a short title hereTesting a short title hereTesting a short title hereTesting a short title hereTesting a short title hereTesting a short title hereTesting a short title hereTesting a short title hereTesting a short title hereTesting a short title hereTesting a short title hereTesting a short title hereTesting a short title hereTesting a short title hereTesting a short title hereTesting a short title here

Testing a short title hereTesting a short title hereTesting a short title hereTesting a short title hereTesting a short title hereTesting a short title hereTesting a short title hereTesting a short title hereTesting a short title hereTesting a short title hereTesting a short title here

Categories
Uncategorized

Tutorial for Artists on how to use a Neural Network — Part 2

Originally published at www.jackalope.tech on April 30, 2018.

Okay so we’ve installed the Ubuntu partition last week, and now we’re going to install the neural network Deep Style. This is where stuff is probably the most difficult. I’m going to equip you with the tools to solve those problems.

What is a CLI?

When you use a program such as your internet browser, or Photoshop you are using a Graphical User Interface. GUI. Before GUI there were CLI . Command Line Interface. A GUI allows you to control a program using buttons. A command line interface allows you to control it using written commands. When you tell Siri to navigate you to your friends house, you are in a way using a modern version of a CLI. Siri is much more sophisticated though. It can listen to your voice and translate that into written words. It can ask for clarification, or intelligently make assumptions about what you actually want. A CLI is much more simple. It’s very literal and strict in how you give it commands. In a way using a CLI is a lot like using a programming language. FEAR NOT! It’s also a lot like asking Siri to do something for you. It’s not really all that hard.

A big advantage of a CLI is that you can create “batch scripts”. Batch scripts are where you automate a bunch of tedious jobs using a programming pattern. I’ll be covering batch scripts more in Part 3 but for now what you need to know is that the name of the language that you write scripts in Linux is called Bash.

There are some people who spend their entire careers mastering Bash. You won’t need to know very much to use the neural network deepstyle though. What you will need to do is use Bash to install the neural network. I’ll explain every step along in the process for you.

If you feel up to it, I highly recommend this interactive tutorial on Codecademy. It’s actually a lot of fun! You use Bash to do things like edit or write poetry. This is not necessary though.

To access a CLI program you need to open your terminal. Ubuntu you can do this by clicking on the start/programs button, and then typing “terminal”. It will search for the default terminal application. Then just press Enter and you’ve got it!

Sometimes throughout these tutorials you’ll see me using a weird looking terminal that looks like an old computer screen. It’s called CRT: Cool Retro Terminal and I decided to use it because I wanted something cool looking for the tutorials but the default terminal as picture above is all you really need and they function basically the same.

The Big Picture

Here are the broad strokes of what we are going to do.

  1. Install the dependencies for the Deep Style Neural Network.
  2. Install the Deep Style Neural Network
  3. Install CUDA to enable GPU processing.

What that means:

Dependencies are pieces of code that a programmer uses to help build their own code. Good code is like lego blocks. You can snap them in and out.
You can think of dependencies like standardized auto parts. If I wanted to build a car I wouldn’t make my own tires or bolts or even the engine from scratch. Programming is all about standing on the backs of giants.

The main dependencies we will download are Torch7, and loadcaffe. They also have their own dependencies. Torch7 will need LuaJIT and LuaRocks. Loadcaffe will need the Google Protocol Buffers. You don’t really need to know what any of these things are, just like you don’t need to know how exactly the spark plug works to install it. In the broad strokes Torch7 is a framework for people to build their neural networks upon. Loadcaffe is an implementation of the Caffe framework in Torch7. Caffe is a neural network framework made by UC Berkley to do image classification and segmentation.

Finally CUDA is a framework by Nvidia that allows programmers to use the GPU to do CPU type work. GPUs are great for doing calculations when they have to process large chunks of data at the same time. Without CUDA your computer will only be able to use the CPU to process the neural network. That will limit the size of images you can produce, and it will take much much much longer to process.

The important take away here is that there is A LOT of people working together to make this stuff possible!

How to get help

Nobody can do this stuff alone. One of the most important skills in programming is being able to ask for help. Stuff is fixing to get pretty technical, so you’re going to need the tools to troubleshoot problems when (not if) they arrive.

Ways you can get help:

  1. You can message me at my facebook page or email me.
  2. You can post an issue to the Deep Style Github page. Github is a website where people share and collaborate on code. Justin Johnson built the implementation of Deep Style that we will be using.
  3. Finally if you need help you can check out Stack Exchange. Stack Exchange is a website for people asking questions. There are two stack exchanges that you will want to check out specifically: AskUbuntu if the issue is related to Ubuntu, or StackOverflow which is for general programming questions.

IMPORTANT INFORMATION:
I don’t want to just tell you where to get help. I want to help you know how to best ask for it. Programming is a technology but it’s also a cultural construct. These aren’t robots you’re sending your questions out to on the internet. They’re communities of people and that means that you want to be respectful of them. Programming communities have their own cultural norms and expectations. You’ll get more help if you prepare for it.

You don’t have to worry too much about messaging me, I’m asking people to do that so I can see what ways I can improve these tutorials. I will say that I’m still early in my developer journey and I’m certainly not a Machine Learning dev, but I’ll help where I can. 🙂

When you post a question on StackExchange, it is important that you make your question concise and to the point. It needs to be a specific question. A good question would be “What hex wrench size do I need to change out the wheel on my Huffy Bike?”. A bad question would be “HELP!??? WHAT DO I DO? IS THIS BROKEN?” Here’s a great guide that goes into further detail.

You can post much more freeform questions to the Github page, but you’ll also want to try and provide as much information as possible. Be prepared to provide a detailed conversation on what you’ve tried, how you’ve tried it etc. Posting screenshots of your terminal results can be really helpful.

I also think that in general it’s a good idea to let people know at the beginning of your request for help your background as an artist etc. This is important because it will let people know your familiarity with the terms they might need to use, but it also means that you’re someone who wants to learn. Programmers love programming. It excites them! They want to share that love with others. Programming isn’t easy though. Programmers know that, so they love it when someone is willing to try and learn something new. Don’t be ashamed! We all started at the same place, and we all get where we are because of the people who went before us.

Time to Install

Install Torch7

Open the terminal and type this in exactly:

cd ~/

CD stands for “change directory. The ~/ stands for “root folder”. That’s the highest level folder of your computer’s memory system.

Next type:

curl -s https://raw.githubusercontent.com/torch/ezinstall/master/install-deps | bash

The CLI will then try to use the “curl” command to grab the installation file. Curl is a program for grabbing a file from a url. It’s short for “see url”. Curl. :/ Your computer probably won’t have curl on it by default. The CLI will give you a prompt asking if you want to install it. Say yes and then let it do it’s thing.

If you had to install curl try running that command again to then actually run the curl process etc.

Once it’s done type in this:

git clone https://github.com/torch/distro.git ~/torch --recursive

You might also have to install git. Just do like curl before, install git, and then when it’s done, re-run the command from above. Git is a neat way for programmers to collaborate on projects together. You don’t really need to know anything about it to use this DeepStyle neural network but if you’re curious I highly recommend this short interactive lesson on Codecademy. Similar to the CLI tutorial mentioned earlier, it’s really cool, and actually teaches you the concepts of git using natural language stuff like poetry and prose editing instead of code.

That will download the newest torch distro from the internet. When it’s done type in this:

cd ~/torch; ./install.sh

That will change your directory to the torch directory and then open a Bash scripting file that someone made to help install the files now that they are on your computer.

Hopefully no problems so far.

Install loadcaffe

Okay first thing is installing the loadcaffe depencies

sudo apt-get install libprotobuf-dev protobuf-compiler

Now you just got to install loadcaffe

luarocks install loadcaffe

Presto! Now it’s time for DeepStyle.

Install DeepStyle

Okay, now it’s time to do the real deal!

First:

cd ~/

Second:

git clone https://github.com/jcjohnson/neural-style.git

Third:

cd neural-style

Fourth:

sh models/download_models.sh

This downloads the learning model. It’s using a Bash script to do this.

Now Test!

th neural_style.lua -gpu -1 -print_iter 1

If it works properly then you’ll see some output like this:

[libprotobuf WARNING google/protobuf/io/coded_stream.cc:505] Reading dangerously large protocol message.  If the message turns out to be larger than 1073741824 bytes, parsing will be halted for security reasons.  To increase the limit (or to disable these warnings), see CodedInputStream::SetTotalBytesLimit() in google/protobuf/io/coded_stream.h.
[libprotobuf WARNING google/protobuf/io/coded_stream.cc:78] The total number of bytes read was 574671192
Successfully loaded models/VGG_ILSVRC_19_layers.caffemodel
conv1_1: 64 3 3 3
conv1_2: 64 64 3 3
conv2_1: 128 64 3 3
conv2_2: 128 128 3 3
conv3_1: 256 128 3 3
conv3_2: 256 256 3 3
conv3_3: 256 256 3 3
conv3_4: 256 256 3 3
conv4_1: 512 256 3 3
conv4_2: 512 512 3 3
conv4_3: 512 512 3 3
conv4_4: 512 512 3 3
conv5_1: 512 512 3 3
conv5_2: 512 512 3 3
conv5_3: 512 512 3 3
conv5_4: 512 512 3 3
fc6: 1 1 25088 4096
fc7: 1 1 4096 4096
fc8: 1 1 4096 1000
WARNING: Skipping content loss
Iteration 1 / 1000
Content 1 loss: 2091178.593750
Style 1 loss: 30021.292114
Style 2 loss: 700349.560547
Style 3 loss: 153033.203125
Style 4 loss: 12404635.156250
Style 5 loss: 656.860304
Total loss: 15379874.666090
Iteration 2 / 1000
Content 1 loss: 2091177.343750
Style 1 loss: 30021.292114
Style 2 loss: 700349.560547
Style 3 loss: 153033.203125
Style 4 loss: 12404633.593750
Style 5 loss: 656.860304
Total loss: 15379871.853590

WOO!

But you aren’t done yet! Now you have to install CUDA. Like I said earlier, if you don’t install CUDA, then you can only use your CPU and that will take foreeeeeeeever.

Install CUDA

To install CUDA you’re going to need to download the right installer. If you followed the first tutorial then you probably installed Ubuntu 16. That means you need CUDA 8. You can find that here.

You’ll want to use these options when picking your installer:

Download and run both of the installers. Reboot the machine. Open the CLI terminal and run

nvidia-smi;

That is to check to make sure everything is installed correctly. It should look something like this:

Sun Sep  6 14:02:59 2015       
+------------------------------------------------------+
| NVIDIA-SMI 346.96 Driver Version: 346.96 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 GeForce GTX TIT... Off | 0000:01:00.0 On | N/A |
| 22% 49C P8 18W / 250W | 1091MiB / 12287MiB | 3% Default |
+-------------------------------+----------------------+----------------------+
| 1 GeForce GTX TIT... Off | 0000:04:00.0 Off | N/A |
| 29% 44C P8 27W / 189W | 15MiB / 6143MiB | 0% Default |
+-------------------------------+----------------------+----------------------+
| 2 GeForce GTX TIT... Off | 0000:05:00.0 Off | N/A |
| 30% 45C P8 33W / 189W | 15MiB / 6143MiB | 0% Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| 0 1277 G /usr/bin/X 631MiB |
| 0 2290 G compiz 256MiB |
| 0 2489 G ...s-passed-by-fd --v8-snapshot-passed-by-fd 174MiB |
+-----------------------------------------------------------------------------+

Now you’re going to install the CUDA backend for torch.

Run this command:

luarocks install cutorch

and then this:

luarocks install cunn

Run this to check and see if the installation has worked:

th -e "require 'cutorch'; require 'cunn'; print(cutorch)"

And then you’ll see an output sorta like this:

{
getStream : function: 0x40d40ce8
getDeviceCount : function: 0x40d413d8
setHeapTracking : function: 0x40d41a78
setRNGState : function: 0x40d41a00
getBlasHandle : function: 0x40d40ae0
reserveBlasHandles : function: 0x40d40980
setDefaultStream : function: 0x40d40f08
getMemoryUsage : function: 0x40d41480
getNumStreams : function: 0x40d40c48
manualSeed : function: 0x40d41960
synchronize : function: 0x40d40ee0
reserveStreams : function: 0x40d40bf8
getDevice : function: 0x40d415b8
seed : function: 0x40d414d0
deviceReset : function: 0x40d41608
streamWaitFor : function: 0x40d40a00
withDevice : function: 0x40d41630
initialSeed : function: 0x40d41938
CudaHostAllocator : torch.Allocator
test : function: 0x40ce5368
getState : function: 0x40d41a50
streamBarrier : function: 0x40d40b58
setStream : function: 0x40d40c98
streamBarrierMultiDevice : function: 0x40d41538
streamWaitForMultiDevice : function: 0x40d40b08
createCudaHostTensor : function: 0x40d41670
setBlasHandle : function: 0x40d40a90
streamSynchronize : function: 0x40d41590
seedAll : function: 0x40d414f8
setDevice : function: 0x40d414a8
getNumBlasHandles : function: 0x40d409d8
getDeviceProperties : function: 0x40d41430
getRNGState : function: 0x40d419d8
manualSeedAll : function: 0x40d419b0
_state : userdata: 0x022fe750
}

Finally you can use GPU to run the neural network!

Check it using this command:

th neural_style.lua -gpu 0 -print_iter 1

if that doesn’t work then…. well something went wrong. Never fear! remember the lessons I wrote above on asking for help. Reach out to people!

Last week was how to install Ubuntu. Next week will be how to run the bash scripts to use DeepStyle. The week after that I’ll go over the actual process I have for taking the imagery created by the program and painting new paintings using Photoshop.

I’ve also got a couple of spin off articles I want to write going over more specific aspects of the process, and some even further afield spinoffs on the meaning this technology has for art at large. If you have any questions or requests for articles let me know! And don’t forget to subscribe to the newsletter!

Categories
Uncategorized

Why Iowa happened and why government websites suck

The democratic caucus fiasco in Iowa is an example of an important thing:

Civic orgs, like the DNC or the government, need to stop using private contracting companies to build their closed source software. Democracy is at stake.


First let me establish some definitions, if you already know this stuff you can skip it, but I just want to make sure this article is accessible. Definitions have also been simplified for the purposes of addressing context.

What does “closed source software mean”?

“Source” refers to the code that a programmer writes to make a computer program. Source code is then generally compiled down into a file that the machine can read to execute the program. This is called “binary code” or you might refer to it as a native app or an executable.

Closed source means that you get the compiled code, but you don’t get the original code that the programmer wrote. This is how most consumers use software, as compiled code. Examples are, the operating system Microsoft Windows 10, apps on the Apple app store, or games on your PC or in a console.

Open source code is code which you have access to the original code. This means you can edit and change your program. You can copy it. You can reuse it for your own purposes.


Open source is necessary for transparent democratic governance.

Let’s start by just agreeing to assume that transparency is a necessary aspect of a democracy. A democracy whose inner workings are inaccessible to the voter might as well be a black box. There’s no way to really hold it accountable or know it’s working.

Open source in civic orgs is the only way to ultimately insure transparency in government. We have reports that candidate Pete Buttigieg paid money (about 40k) to the company Shadow, funded by superPac ACRONYM (fire the show writer please). Now we don’t have to immediately assume this is corruption, this could be innocent. But it’s kind of hard without being able to see the code ourselves and audit things.

There are ways to do this with open source projects. First the source is directly available so people can just see it. Not only that but you can verify your vote!

The open source blockchain project Hyperledger was used to make the West Virginian experiment in smartphone voting, Voatz.

Helios provides a voting server system that allows voting by email, that uses hashes to confirm your vote and keep your vote anonymous at the same time. Helios is obviously not a good pick, design wise, for national elections but it’s encryption and verification method certainly are.

If transparency is so important to democracy then why isn’t the infrastructure being developed for democracy also being transparently developed in the most transparent way possible?

Open source is extremely common and yet most civic org code is closed source. Why?

It’s simple.

Money

1.

Closed source companies sold fearful portents about the security of open source in order to secure contracts with civic organizations.
Security is not an issue with open source. In fact it is generally speaking more secure than closed source software. The vast majority of online servers are Linux based. Microsoft itself is one of the biggest contributors to Linux and now runs many of their servers on Linux. The preferred OS for hacker and crypto types is a Linux variant called Kali.

The reason why is simple. A team of 3 or even 300 devs are nothing compared to what the eyeballs of 30,000 can spot. Bugs will happen. But like a body with a healthy immune system, the only way to deal with potential problems is exposure and adaption. Open source projects can’t hide behind obfuscation, which is more often a crutch than anything else. Why were we told anything different? As I said, it’s about money. But this isn’t the only way it’s about money.

2.

Leaders at civic orgs have been incentivized to buy from private contractors. This is a problem for a number of reason:

2.1

It strips internal departments of their tribal knowledge, and reduces the investment in improving and training staff.

Let’s say a contractor came in and made a web app. You might do this because it’s more cost effective in the short term to bring in an expensive but short term team, that already knows how to make the web app. They just come in and make the app. Then some of them might stay awhile to train up some internal staff for hand off, but then from then on the web app is largely just being maintained. This is essentially a form of deskilling.

This is a problem because there’s only so much you can document and teach. A lot of experience is gained through the process of building something yourself. And employees are not static, they have growth potential. Dividing out efforts between internal and external teams means losing a substantial part of the people growth and it disincentivizes investing further growth in internal employees because they largely are relegated to a role of keeping the ship on course.

2.2

Private external contracting companies do not have an incentive to open source their work, generally speaking. They are a private company after all.

2.3

Between these first two you get this third issue. Self dealing behavior.

That Pete Buttigieg payment to Shadow? Well it might interest you to learn that the Tara McGowan, the owner of ACRONYM, has a husband named Michael Halle, and he’s a senior strategist for Pete…

We can’t prove anything happened here obviously but the power to award contracts has long been used as a tool of “machine politics” and graft. There is value created in the process of building things (the cultivation of tribal knowledge and the development of libraries and systems to reuse) and that value is externalized from the public space and internalized to the private space. Make profits private, make losses public. The old Bank Bailout strategy.

2. 4

Contracts are a way for politicians to benefit those they like and people who will support them. Often rich people. Often superPacs. Lawerence Lessig makes the argument for how superPacs corrupt the system in his TED talk here. The same basic principle applies.


Open Source solves problems you didn’t even know you had

Besides increasing transparency/security there are additional bonuses to shifting our government from private closed source projects to public open source projects.

Code Reuse:

Sure, growing an internal team long term is going to be more expensive upfront but it’s an investment with a very very long tail.

A project which is used in Iowa can be taken and repurposed in another caucus state. Or can be repurposed by a caucus like structure in another country! All programming and computer science is like this. It’s people building on the work who built on the work of other people. The initial investment in ARPAnet, and research into computer processing was all government funded, as a big upfront investment that has had a long tail of returns. We need to reinvest in our technological infrastructure, not just so that crap like Iowa doesn’t happen but so that there is a rising tide to lift all boats. Economists Mariana Mazzucato makes this argument in her book The Entrepreneurial State.

Bottom up, grassroots experiments in democracy:

There are lots of experiments that can be attempted at local levels with the possible overflow into larger levels, bottom up style, rather than top down.

Remember how I mentioned Helios and Voatz earlier? Helios allows for mathematically certain verification that your vote was counted. But it also allows for alternatives to First Past the Post (the traditional voting you know of), such as Ranked Choice voting. This is simply the tip of the iceberg.

I recommend checking out these two other projects, purely from a design perspective, because they show that there is a lot of of further changes to the system that could be made to improve it.

Kialois an argument and discussion aid tool that structures conversations in ways that are meant to create proper citation, avoid logical fallacies, and map the difference of opinion across many levels of an issue.

Loomiois an open source “decision making platform”. Think of it like slack but for getting stuff decided instead of endlessly flooded with gifs. It operates on principles of discussion/debate that were developed by organizers of Occupy Wallstreet.

Provide a method to improve democracy:

Think of democracy as a code project. If you’re reading this article you might not know much about code, but you can observe these basic points I’m going to mention in a lot of human systems around you, music scenes, the “vibe” of a relationship, the evolution of language and how we pass on knowledge in communities.

Code projects, particularly open source ones, usually start with someone trying to solve a problem. They go at solving that problem, and they write this initial program. That program is going to have certain decisions they made for certain reasons that at the time made sense but maybe don’t now. Or perhaps there were bugs in the code, so the developers patched it up with duct tape and chewing gum, which works but if you push the wrong lever or button the whole thing just shivers and shakes and falls apart in a heap on the floor.

In tech circles this is referred to as “tech debt”. This is obviously a problem. It’s a problem not just with programming code, it’s a problem with our system of governance in general. Think of laws as similar to programming code. Did you know that there’s nowhere in the constitution where it is stated that the job of the Supreme Court is to determine constitutionality?

That’s because the Founders didn’t specify that…why? I have no clue. But the constitution is the first of it’s kind in modern history, an explicitly democratic republic. It’s not surprising that maybe they didn’t get it perfect the first time. I mean, they did immediately amend it, after all. It’s a document which was built to evolve.

But the problem is that it’s the beta. Or as devs say “devving in prod”. There’s been a lot of innovations in democracy in the last 250 years. Alternative choice! Parliamentary systems! Mathematical models to prevent Gerrymandering!

But what we have instead is the beta. And it’s old and it’s crusty and the chewing gum is starting to get brittle. We need real solutions.

A big part of fixing this is fixing our infrastructure. And open source is key to that.


One last thing:

This won’t be easy.

Open source is not known for being user friendly. Neither is politics.

Those who make open source are also the ones who use it so it tends to trend more towards technical users. This is much less true than it used to be. But it still is a little true. But this is not necessarily a bad thing.

Technology should not be a black box. Just as civics class is a necessary part of being a citizen, technical knowledge is a necessary part of being a digital citizen. This isn’t going to be easy but… like really when has democracy ever been “easy”, right?

Democracy can’t happen without you, as User, and Author. Do no let an elite political class define the system you use. Government for The People, by The People, Author and User one and the same.

mr. barnie sanders saying hello
Categories
Uncategorized

How Artists can use Neural Networks to make Art.

original-publish-date: "2017-03-07"

Let’s start with the basics. What are neural networks? Neural networks are a computational approach to solving problems that is modeled off the structure of the human brain. Rather than taking an input and giving a single output, (x+y=z) they take in a large set of inputs and run those inputs through a large set of nodes/neurons that transform those inputs and give a single output (a,b,c,d, e, is output as “Cabde). You throw a bunch of data through the nodes and you’re not going to get a very coherent output. This is how neural networks are different than traditional computing. They learn from their mistakes. You throw in data, you get out junk. You tell the computer that it’s output is warmer or colder, and then it tries again. It does this millions of times until it gets a balance of nodes that gives a rough approximation of a “correct”answer. This is super useful for the kinds of tasks that humans have usually excelled out while computers have not. Things like categorization, object recognition, speech recognition, or more “intuitive” guesswork.

Here’s a good video explaining the subject:​

Also a shoutout to @stephencwelch for such a great video explaining the subject. One of the “human” things that neural networks are making progress in is art. Let me introduce you to DeepStyle:

Click the image above to check out the Style Transfer Neural Network research paper

Essentially what Deep Style does it has been trained to recognize objects and texture. You give it a photo, and you give it a painting, and it makes the photo look like it was painted in the style of the painting. You’ve probably seen the apps in the App Store that use this technology, calling the different painting effects “filters”. This process is very similar to a Google project called “Deep Dream” which you can read about here:

Google Deep Dream

Deep Style isn’t just a Photoshop. It’s a lot more intelligent than that, but like Photoshop it should be looked at as a tool for artists. The development of art is intrinsically linked to the development of technology. Impressionism came out of new scientific discoveries in optics. The invention of amplified electric instruments lead to rock’n’roll.Technology creates and defines the canvas, but we have to decide what to put on there.

I will admit that having been trained in a traditional oil painting fine arts tradition kind of has given me a certain amount of leeriness towards “cheating”. As I’ve gotten older though I learned about the long history of “mechanical reproduction” in art. Going back to things like the camera obscura, all the way to modern “photobashing” people have been finding ways to “cheat” and produce work faster and more accurately. Ever seen those super impressive videos on YouTube where people draw those ultra realistic portraits just by drawing across the board? That’s actually a mechanical reproduction technique not unlike tracing. For the longest time I couldn’t figure out how people could just move their hand linearly across the board filling in the details like a printer, but that’s actually part of the trickery of the time lapse.​

So Deep Style can be used as a tool, like any other artistic tool. I wanted more flexibility than downloading an App with presets would give me, so I partitioned my computer hard drive, and installed Linux to run the neural network. You can download an implementation of the neural network on GitHub  here .

A friend of mine, Rachel Garza, who works at Unity. Go follow her twitter! @fiendinacloud

I’m not a computer scientist. I’m an just an artist who finds tech interesting. Right now, neural networks are so niche that they’re mostly only used and understood by Silicon Valley engineers and computer science academics. I think the basic concept of a neural network is actually quite intuitive, it’s modeled after our own brain structure after all, but despite the intuitiveness of it, it still has a bit of a barrier to entry for non-technically inclined people. I hope by sharing my experiences and knowledge that I will help other artists understand neural networks and find cool ways to implement them in their own workflow.

First a couple of terms:

Content image: This is the photo you want to transform. If I were trying to make a dog look like it had been painted by Monet, the dog would be the content image.

Style image: This is the image you are deriving the style from. So in the case of the previous example, Monet would be the style image.The possibilities left to experiment with neural networks, even just Deep Style itself are massive, but I’ll focus on one particular set of techniques I’ve been experimenting with. I call these paintings “neural fusion paintings”

Basics of the technique: I run an image through a neural network under a bunch of different parameters, and using different source style images. Sometimes from the same artist, sometimes multiple artists. Then I layer those styled images over the original photo and use a layer Mask in Photoshop to selectively reveal and hide different parts of the style images. Then I have a final layer where I touch up some of the details and blend different parts of the image.

Phil is a cool guy, developer, and creative director in Austin TX. He made the game Battlesloths which you should def check out!

This neural fusion technique is useful for a couple of reasons. First, I have a pretty hefty computer (I also do VR art and development), but even my computer has trouble processing these images. Neural networks take a massive number of simultaneous computations. This means they need a really good graphics card. I have a GTX 970 with 3.5 gigs of VRAM and that means I can get, with optimizations, output images that are about 600–800 pixels in width.

People have come up with some solutions, like slicing up the images into smaller pieces, and then applying the neural network to each piece and then sewing them back together, but this technique results in a “flat” look that loses a lot of the object recognition skills that make Deep Style so convincingly like a painting.

With the neural fusion technique though, some of the lack in resolution can be disguised through your manual blending layer in Photoshop. Another advantage to the neural fusion technique is that the hand-painted detail layer in Photoshop can make up for some of the lack of information resolution that the neural network can access on a smaller graphics card. The bigger the graphics card, the higher res the image that it can hold in vRAM, and therefore the more texture details it can extract from it. If you resize a picture of a person down to 100px by 100px then the photo is so small that you can’t really meaningfully extract any useful information about the facial recognition or text etc. It’s just a bunch of splotchy color squares. The apps you can get in the App store run on servers that are probably more powerful than your home computer, which means they can do some pretty rad stuff even if you don’t get the same control as a home installation.

The very talented artist, Fredrick Nolting has done some awesome work using the output from apps as a starting place for his expressionistic paintings:

Seriously though, check out his blog

There are lots of possibilities, and as GPU power goes up, the possibilities for artists will go up. This is a rich vein to explore. It won’t just be an Instagram filter, an automatic conversion of a photograph. It will provide a tool to transform visual material for the artist to sculpt and bend to their ends.

There are two aspects to “painterly” graphical representation that need to be taken into account. Noise and Specificity. Real life is naturally noisey. Light and color have very complex interactions with the surface of a material and the way they bounce around a room. There’s lots of tiny variations from speck to speck. When we say that a piece of CGI look “fake” what we’re usually grabbing on to is an insufficiently complex simulation of light on the surface, resulting in a simplistic, homogeneous “plastic” reflection of light. (remember, polymers are basically just a repeating chain of molecules so the way light interacts with it is going to be fairly consistent compared to the way it would interact with a complex wabi-sabi organic surface like skin).

“Un-noisey” rendering style might be preferable though! As argued by Scott McCloud, the emotional strength of art is often tied to it’s ability to simplify and abstract the world.

The process of running images through Deep Style naturally lowers their surface noise. Even though the resulting images can sometimes be harder to make out, or might look more chaotic, on a pixel by pixel level they’re generally more consistent. You can actually do this in Photoshop to some degree using it’s “Artistic” or “Facet” filters but this will end up looking like a filter because they lack the second element.

Scott McCloud is probably one of the people I owe the most to my artistic thought process.

Specificity is in some ways at odds with decreasing noise. As you decrease the overall noise, you often decrease distinction, the sharpness dividing a foreground object and it’s background. It sort of smooths out details and makes everything a little “smudgey”. This makes a picture look less like a real painting because it looks programmatic. It essentially created without reference to the object depicted. A Photoshop filter doesn’t know that it’s applying an Artistic filter to a face, or a dog, or a landscape. It just sees a grid of colors. Neural networks are able to bring greater specificity in the way they denoise and apply texture which creates a more accurate illusion of being painted by hand. It seems to carry with it intention rather than being the result of an anonymous algorithm.

Fox is a really cool guy who makes VR Games you’ve actually probably played! Follow him on Twitter @FoxBuchele

The places that the neural network have the most difficulty with, and consequently suffer the most at lower resolution are in facial features and fine edges, areas of high specificity. This is another advantage of the neural fusion technique. Details that might otherwise get smudged out, like individual hairs, eyelashes, or pupils, can have their specificity brought back in through a hand painted detail Photoshop layer.

Common issues are smudges between surfaces (a face might kind of “bleed” into the background behind it) or a rippling effect where there should be a consistent flat or curved surface (but the neural network doesn’t know that it’s a surface. While your eyes know what a hand is, and the curve of a hand, the neural network is just working with a bunch of pixels and trying to find the average) You can selectively reveal some of the original photo to help here but the texture and color will often not match properly and look a little weird because photographs are generally higher noise than the intended painted look. Using some of the Artistic filters in Photoshop along with hand painting generally can help out with this issue if you do choose to use the photograph directly in the painting. Having figure drawing experience is a big help here, even if in many ways the bulk of the work is being taken care of for you by the neural network.

There’s more I could dive into here, like the technical details of setting up the neural network for non-technical people etc but I think I’ll bring this article to a close for now. I would love feedback! If you’re an artist working with neural networks, let me know! If there’s something unclear or that you’d like to know more about ask me! Here’s a bonus gif to show how the layers are built up:

Below you can also see some other portraits I’ve made. I’m going to be expanding on this process very soon, branching out from portraits, and exploring some additional techniques, so be sure to subscribe to the newsletter and stay tuned!

Hello world!

Time for a fresh start. Separating out out my art/tech/design/development musings from my professional work. This gives me more freedom to update on matters not strictly related to my job.

I’ve been learning things faster than I could get it down on paper. I will repost some of the old articles here for posterity but it’ll be mostly new stuff. The old stuff often times became out of date by the time I posted it.

I’m embracing a more casual and streamlined presentation style. Less mental overhead required and fresher more relevant content.