Tensorflow中解析LSTM实现方式 – PLSTM

Tensorflow中解析LSTM实现方式

结果

下面是在高分辨率采样的鉴频结果:深蓝色的PLSTM收敛速度比GRU(浅蓝色)快。

实现方式

下面以一种即插即用的方式实现PLSTM,如果你想在你的其中一个模块中使用它你可以将LSTMCell/GRUCell 转换为 PhasedLSTMCell。

PLSTM核心

tau = vs.get_variable(
        "T", shape=[self._num_units],
        initializer=random_exp_initializer(0, self.tau_init), dtype=dtype)

    r_on = vs.get_variable(
        "R", shape=[self._num_units],
        initializer=init_ops.constant_initializer(self.r_on_init), dtype=dtype)

    s = vs.get_variable(
        "S", shape=[self._num_units],
        initializer=init_ops.random_uniform_initializer(0., tau.initialized_value()), dtype=dtype)
        # for backward compatibility (v < 0.12.0) use the following line instead of the above
        # initializer = init_ops.random_uniform_initializer(0., tau), dtype = dtype)

    tau_broadcast = tf.expand_dims(tau, dim=0)
    r_on_broadcast = tf.expand_dims(r_on, dim=0)
    s_broadcast = tf.expand_dims(s, dim=0)

    r_on_broadcast = tf.abs(r_on_broadcast)
    tau_broadcast = tf.abs(tau_broadcast)
    times = tf.tile(times, [1, self._num_units])

    # calculate kronos gate
    phi = tf.div(tf.mod(tf.mod(times - s_broadcast, tau_broadcast) + tau_broadcast, tau_broadcast),
                 tau_broadcast)
    is_up = tf.less(phi, (r_on_broadcast * 0.5))
    is_down = tf.logical_and(tf.less(phi, r_on_broadcast), tf.logical_not(is_up))

    k = tf.select(is_up, phi / (r_on_broadcast * 0.5),
                  tf.select(is_down, 2. - 2. * (phi / r_on_broadcast), self.alpha * phi))

然后kronos gate可以直接应用到单元格中:

c = k * c + (1. - k) * c_prev
        m = k * m + (1. - k) * m_prev

PhasedLSTMCell具有相同的参数设置以及LSTMCell plus,下面报告默认参数。

gete关断期的斜率:

alpha=0.001

r_on的初始值

r_on_init=0.05

tau的初始采样参数

tau_init=6.

本站文章除注明转载外,均为本站原创或编译
转载请明显位置注明出处:Tensorflow中解析LSTM实现方式 – PLSTM