wie man ein frühes Anhalten im Tensorflow implementiert

Hier ist meine Implementierung des frühen Stopps, den Sie anpassen können:

Das vorzeitige Anhalten kann in bestimmten Phasen des Trainingsprozesses angewendet werden, z. B. am Ende jeder Epoche. Speziell; in meinem Fall; Ich überwache den Testverlust (Validierungsverlust) bei jeder Epoche und nachdem sich der Testverlust nach 20 Epochen (self.require_improvement= 20) Nicht verbessert hat, wird das Training unterbrochen.

Sie können die maximalen Epochen auf 10000 oder 20000 oder was immer Sie wollen (self.max_epochs = 10000) Einstellen.

  self.require_improvement= 20
  self.max_epochs = 10000

Hier ist meine Trainingsfunktion, bei der ich das frühe Stoppen nutze:

def zug (selbst):

# training data
    train_input = self.Normalize(self.x_train)
    train_output = self.y_train.copy()            
#===============
    save_sess=self.sess # this used to compare the result of previous sess with actual one
# ===============
  #costs history :
    costs = []
    costs_inter=[]
# =================
  #for early stopping :
    best_cost=1000000 
    stop = False
    last_improvement=0
# ================
    n_samples = train_input.shape[0] # size of the training set
# ===============
   #train the mini_batches model using the early stopping criteria
    Epoch = 0
    while Epoch < self.max_epochs and stop == False:
        #train the model on the traning set by mini batches
        #suffle then split the training set to mini-batches of size self.batch_size
        seq =list(range(n_samples))
        random.shuffle(seq)
        mini_batches = [
            seq[k:k+self.batch_size]
            for k in range(0,n_samples, self.batch_size)
        ]

        avg_cost = 0. # The average cost of mini_batches
        step= 0

        for sample in mini_batches:

            batch_x = x_train.iloc[sample, :]
            batch_y =train_output.iloc[sample, :]
            batch_y = np.array(batch_y).flatten()

            feed_dict={self.X: batch_x,self.Y:batch_y, self.is_train:True}

            _, cost,acc=self.sess.run([self.train_step, self.loss_, self.accuracy_],feed_dict=feed_dict)
            avg_cost += cost *len(sample)/n_samples 
            print('Epoch[{}] step [{}] train -- loss : {}, accuracy : {}'.format(Epoch,step, avg_cost, acc))
            step += 100

        #cost history since the last best cost
        costs_inter.append(avg_cost)

        #early stopping based on the validation set/ max_steps_without_decrease of the loss value : require_improvement
        if avg_cost < best_cost:
            save_sess= self.sess # save session
            best_cost = avg_cost
            costs +=costs_inter # costs history of the validatio set
            last_improvement = 0
            costs_inter= []
        else:
            last_improvement +=1
        if last_improvement > self.require_improvement:
            print("No improvement found during the ( self.require_improvement) last iterations, stopping optimization.")
            # Break out from the loop.
            stop = True
            self.sess=save_sess # restore session with the best cost

        ## Run validation after every Epoch : 
        print('---------------------------------------------------------')
        self.y_validation = np.array(self.y_validation).flatten()
        loss_valid, acc_valid = self.sess.run([self.loss_,self.accuracy_], 
                                              feed_dict={self.X: self.x_validation, self.Y: self.y_validation,self.is_train: True})
        print("Epoch: {0}, validation loss: {1:.2f}, validation accuracy: {2:.01%}".format(Epoch + 1, loss_valid, acc_valid))
        print('---------------------------------------------------------')

        Epoch +=1

Wir können den wichtigen Code hier fortsetzen:

def train(self):
  ...
      #costs history :
        costs = []
        costs_inter=[]
      #for early stopping :
        best_cost=1000000 
        stop = False
        last_improvement=0
       #train the mini_batches model using the early stopping criteria
        Epoch = 0
        while Epoch < self.max_epochs and stop == False:
            ...
            for sample in mini_batches:
            ...                   
            #cost history since the last best cost
            costs_inter.append(avg_cost)

            #early stopping based on the validation set/ max_steps_without_decrease of the loss value : require_improvement
            if avg_cost < best_cost:
                save_sess= self.sess # save session
                best_cost = avg_cost
                costs +=costs_inter # costs history of the validatio set
                last_improvement = 0
                costs_inter= []
            else:
                last_improvement +=1
            if last_improvement > self.require_improvement:
                print("No improvement found during the ( self.require_improvement) last iterations, stopping optimization.")
                # Break out from the loop.
                stop = True
                self.sess=save_sess # restore session with the best cost
            ...
            Epoch +=1

Hoffe es wird jemandem helfen :).

ValidationMonitor wird als veraltet markiert. es wird nicht empfohlen. Sie können es aber trotzdem verwenden. Hier ist ein Beispiel, wie man eins erstellt:

    validation_monitor = monitors.ValidationMonitor(
        input_fn=functools.partial(input_fn, subset="evaluation"),
        eval_steps=128,
        every_n_steps=88,
        early_stopping_metric="accuracy",
        early_stopping_rounds = 1000
    )

und du kannst es selbst umsetzen, hier meine Implementierung:

          if (loss_value < self.best_loss):
            self.stopping_step = 0
            self.best_loss = loss_value
          else:
            self.stopping_step += 1
          if self.stopping_step >= FLAGS.early_stopping_step:
            self.should_stop = True
            print("Early stopping is trigger at step: {} loss:{}".format(global_step,loss_value))
            run_context.request_stop()

Seit der TensorFlow-Version r1.10 stehen in early_stopping.py frühe Stopp-Hooks für die Estimator-API zur Verfügung (siehe github ).

Zum Beispiel tf.contrib.estimator.stop_if_no_decrease_hook (siehe docs )