Neural Networks

Earlier this year (towards the end of my un­der­grad) I started to record some notes on ma­chine learn­ing ap­pli­ca­tions and the datasets that power them. While I did find some in­ter­est­ing ar­ti­facts (see the FERET data­base) and read some good ar­ti­cles (such as The Body and the Archive), I never man­aged to re­ally train and use a neural net­work in prac­tice.

I did ex­per­i­ment with some pre-built net­works, such as DCGAN by and char-rnn, but I did­n’t re­ally un­de­stand what was hap­pen­ing un­der the sur­face. More im­por­tantly, I could­n’t mod­ify these net­works to change their out­puts (apart from tweak­ing some hy­per­pa­ra­me­ters).

The fo­cus of the next few months is go­ing to be to change that. As with ear­lier pro­jects, I will be tak­ing notes here as I go along.

Ideas for a work­shop

Sheena sug­gested I might run a work­shop with Interpolate on the sub­ject of ma­chine learn­ing. This comes af­ter a dis­cus­sion at Designing a Feminist Alexa. We agreed that there seems to be an aw­ful lot of mag­i­cal think­ing among hu­man­i­ties-folk about what ma­chine learn­ing is. To me, it of­ten feels like these de­bates are float­ing in thin air. Arguments seem to be based more on Black Mirror episodes and Ted Talks than the pa­pers where the work­ing mech­a­nisms of neural net­works are be­ing de­vel­oped.

The goal of the work­shop would be to dis­pel some of those myths, and give peo­ple some low-level un­der­stand­ing of what they’re talk­ing about. I’m propos­ing to do this by hav­ing a group of peo­ple train a neural net­work by hand — us­ing noth­ing but pen, pa­per, and maybe a ba­sic cal­cu­la­tor. Over the course of a few hours, we’d build up a net­work us­ing string and in­dex cards pinned to the wall. Once the net­work is suf­fi­ciently trained, we’d use it to gen­er­ate some kind of out­come.

I’m imag­in­ing it like the scene in Apollo 13 where en­gi­neers in mis­sion con­trol are work­ing out or­bital me­chan­ics us­ing slide rules.

The main ques­tion is: What kind of task do we train this net­work on? It has to be sim­ple enough to be ac­com­plished by a small net­work with lim­ited com­put­ing power, yet com­plex enough to keep peo­ple in­ter­ested.

November 22, 2018: Some more spe­cific work­shop ideas

The learn­ing al­go­rithm is taken straight out of Rosenblatt (1958):

$$w_{i,j}^{\text{next step}} = w_{i,j} + \eta (y_i - \hat{y}_j)x_i$$

Once the model is trained, it can be eval­u­ated:

Monday, December 17

Notes from run­ning the work­shop my­self:

Things that my go in the reader