Elton Y. Wong

Now this looks interesting! The amp signal doesn’t look simply like a “clipped” version of the clean tone. Whatever function is transforming the direct signal into the mic’d signal looks pretty complicated. To the eye, the clean waveform looks sort of truncated in places, but with a higher-frequency signal superimposed on it. This also makes sense, ultimately: the design of the amplifier doesn’t simply boost and then clip the signal. Rather, clipping happens in stages, and there are EQ adjustments at almost every step (for example the cathode bypass resistors of the gain stages are rather small so that low-frequency response is limited). This is part of what makes one guitar amp sound different from another.

 

Design of the Neural Net

 

So, with our data selected, how will we design our algorithm? Let’s take a step back and consider what we’re trying to do. Our input is a one-dimensional array of values between -1 and 1, with a length equal to 44,100 * length of the recording in seconds. Our output will also be a one-dimensional array within the same range of values, and should be about the same length (if we lose some data because we have a window, that’s ok). So, we want our algorithm to learn a function that converts an input vector into an output vector. We won’t worry about real-time signal processing for now, and given that neural nets are so computation-heavy, this will probably remain a secondary concern.

 

But let’s take this a little further. How long should the input vector be? Ideally, we want to be able to convert a recording of a clean guitar at any length to the proper output. Thus, we shouldn’t rely on pre-defining the length of the input vector. With that in mind, let’s ask ourselves – what portion of the input signal at time t do we expect to have an effect on the output signal at time t?  This will tell us how to structure the data we feed into the algorithm. Obviously, anything that the guitar does in the future can’t affect what the amp does now. But how far in the past should we let the algorithm look?

 

This is not a trivial question. A totally straightforward clipping algorithm would not need any history at all – it would simply have a floor and ceiling of the waveform, and if the instantaneous value of the input signal exceeded those, the algorithm would simply return the value directly within the floor or ceiling until the wave fell back within the defined limits. We know from visualizing the waveform above that the amp isn’t doing anything this simple.

 

In addition, we should take into account what we know about how amplifiers work. There are various ways in which an amp’s instantaneous state would be dependent on the path it took to reach that state. For instance, the speaker has some elasticity and would store energy from its previous trajectory. The power supply is non-linear, and would be expected to “sag” somewhat after a large impulse (especially since this amp uses a tube rectifier). Even the iron core of the output transformer has hysteresis loss from the fluctuating current. The fact that we have a microphone recording a speaker cone in a room makes things even worse: the signal at the microphone’s diaphragm would contain at least some sound that has been bouncing around the room for some appreciable fraction of a second. Again, we are asking a lot of our algorithm.

 

Calculating the time frame for all these effects would be beyond the scope of this little experiment, so I decided to simply assume that the signal presented to the amp in the last 1/10th of a second would be fed into the algorithm. This window would theoretically allow frequencies down to 10Hz to be detected, and this is far below any frequencies generated by a guitar. I would “cheat” and allow the algorithm to see a small number of points into the future, to account for any latency/lag in the signal coming from the amp (this was probably not useful, but it shouldn’t hurt either).

 

So, now we know that for every moment (44,100 moments per second), we want the neural net to produce one float value representing the value of the recorded amp. The input will be 4000 points mostly preceding that moment.

 

def make_xy(direct, amp, start_ind = 0, end_ind = 10000, window = 600, future = 10):

    if not end_ind + future < len(direct):

        print('ERROR: Indices extend beyond data')

        return 0

    if not len(direct) == len(amp):

        print('ERROR: direct signal and amp signal must be of equal lengths')

        return 0

    time1 = time.time()

    start_ind = max(start_ind, window)

    list_X = []

    list_Y = []

    for k in range(start_ind, end_ind):

        list_X.append(direct[(k - window + future) : k + future ])

        list_Y.append(amp[k])

   

   

    X_arr = (np.array(list_X, dtype = 'float32'))

    Y_arr = np.array(list_Y, dtype = 'float32')

    time2 = time.time()

    print('Time to make XY: {0:.2f}'.format(time2 - time1))

    return X_arr, Y_arr

 

window = 4000

future = 300

 

X, Y = make_xy(direct, amp, start_ind = 250000, end_ind = 800000, window = window, future = future)

 

X_test, Y_test = make_xy(direct, amp, start_ind = 800000, end_ind = 1500000, window = window, future = future)

 

[EDIT: with the newer version of Keras, I would certainly use TimeseriesGenerator instead of making this array in memory.]

​

Now that we have our training data (about 12 seconds of guitar playing from the beginning of the recording), let’s define and build the neural net:

 

from keras.models import Sequential, Model

from keras.layers import Dense, Dropout, Activation, Input

from keras.utils import np_utils

from keras import backend as K

K.set_image_dim_ordering('th')

 

model = None

model = Sequential()

​

model.add(Dense(256, activation = 'relu', input_shape=(window,)))

model.add(Dropout(0.5))

model.add(Dense(256, activation = 'relu'))

model.add(Dropout(0.5))

model.add(Dense(256, activation = 'relu'))

model.add(Dropout(0.5))

 

model.add(Dense(1, activation = 'tanh'))

 

model.compile(loss = 'mean_squared_error',

              optimizer = 'RMSprop',

              metrics = ['accuracy']

              )

 

history = model.fit(X, Y,

                    batch_size = 256, epochs = 2, verbose =1)

 

Our neural net will have dense layers – every input value will be connected to every neuron in the next layer. Why not use convolutions? Well, we are dealing with a continuously variable input vector that can have any frequency (because we might play any note of the guitar), so we have no reason to expect that there will be some reasonable number of identifiable “motifs” that will provide us with useful information, the way that all frontal photographs of cat faces look sort of the same. Why not something like an LSTM, which is commonly used to model time series data? Answer: we are not trying to predict the future of the amplifier signal based on the past of the amplifier signal – rather, we are trying to learn the function that transforms the clean signal into the amp’s distorted signal, and we are even willing to look into the future of the training data a little. This is a regression problem.

 

There is nothing magical about using three layers of 256 nodes each, or the proportion of dropout – these were chosen as reasonable prior parameters. The choice of tanh for the output is dictated by the data – the output signal varies between -1 and 1, so this is a natural choice of activation function. Mean squared error is a natural choice for what is basically a regression problem. Let’s train the model:

 

 

Epoch 1/2

550000/550000 [==============================] - 6s - loss: 0.0025 - acc: 2.8909e-04    

Epoch 2/2

550000/550000 [==============================] - 6s - loss: 0.0015 - acc: 2.8909e-04 

 

Training took just a few seconds on my computer. Hrm, this seems reasonable enough, but how does it sound? To see, we will use our test data, which the model has never seen. We’ll make predictions and compare with the real amplified signal. First, here is the real amplifier: