Optuna


Quick intro to Optuna - 5 minutes of fame

4 February 2022 @ CzodrowskiLab

Aishvarya Tandon

  • PhD student - CzodrowskiLab / Waldmann group
  • Contact information: Email, GitHub

In the following Jupyter notebook, I will introduce you to Optuna, and why I like it and use it in my workflows.

The following work is based on my opinion. Credits of all the software used and demonstrated belongs to their respective authors and the community.

License: MIT

What is Optuna?

Optuna is a hyperparameter optimization software. It is open-source and more information can be found:

  1. On developer's website
  2. On developer's GitHub page
  3. In their publication (Akiba, Takuya, et al. "Optuna: A next-generation hyperparameter optimization framework." Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining. 2019.)

Toy problem: Models to predict a molecule's average molecular weight from its 2D fingerprint.

Let us prepare data and input strategy which we will use repeatedly in advance.

I will import a subset of randomly selected 50000 compounds from CHEMBL 29

In [1]:
import pandas as pd
df = pd.read_parquet('Shortlisted_Chembl29-4Feb2022.parquet.gzip')
df.info()
df.head()
<class 'pandas.core.frame.DataFrame'>
RangeIndex: 50000 entries, 0 to 49999
Data columns (total 3 columns):
 #   Column     Non-Null Count  Dtype 
---  ------     --------------  ----- 
 0   chembl_id  50000 non-null  object
 1   InChIKey   50000 non-null  object
 2   Smiles     50000 non-null  object
dtypes: object(3)
memory usage: 1.1+ MB
Out[1]:
chembl_id InChIKey Smiles
0 CHEMBL424856 KNZSNTNUOIRWRE-UHFFFAOYSA-N CC1(F)CC(F)(F)c2nn(-c3c(Cl)cc(C(F)(F)F)cc3Cl)cc21
1 CHEMBL3792491 NTQFMRZDTVXICG-UHFFFAOYSA-N CCN(CCCCCCNC(=O)C=Cc1ccc(O)c(OC)c1)Cc1ccccc1OC
2 CHEMBL3900710 IRTOHFGPHMCDRK-UHFFFAOYSA-N Cc1cc(N2C(=O)c3cc(-c4ccn(C)c(=O)c4)n(C4CC4)c3C...
3 CHEMBL35988 YBIWOQATDVKIPT-UHFFFAOYSA-N CC(O)c1cccc(C(C#N)C(=N)Sc2ccccc2N)c1
4 CHEMBL3452898 MMMKGEGGKRMNHL-UHFFFAOYSA-N Cn1ccc(C(=O)N=c2[nH]c(-c3ccc4c(c3)CCO4)cs2)cc1=O

Let us use RDKit's Descriptor module to calculate average molecular weight of these compounds

In [4]:
from rdkit.Chem import AllChem as Chem
from rdkit.Chem import Descriptors, MolFromSmiles

df['MolWt'] = df['Smiles'].apply(lambda x: Descriptors.MolWt(MolFromSmiles(x)))
df.head()
Out[4]:
chembl_id InChIKey Smiles MolWt
0 CHEMBL424856 KNZSNTNUOIRWRE-UHFFFAOYSA-N CC1(F)CC(F)(F)c2nn(-c3c(Cl)cc(C(F)(F)F)cc3Cl)cc21 389.126
1 CHEMBL3792491 NTQFMRZDTVXICG-UHFFFAOYSA-N CCN(CCCCCCNC(=O)C=Cc1ccc(O)c(OC)c1)Cc1ccccc1OC 440.584
2 CHEMBL3900710 IRTOHFGPHMCDRK-UHFFFAOYSA-N Cc1cc(N2C(=O)c3cc(-c4ccn(C)c(=O)c4)n(C4CC4)c3C... 500.986
3 CHEMBL35988 YBIWOQATDVKIPT-UHFFFAOYSA-N CC(O)c1cccc(C(C#N)C(=N)Sc2ccccc2N)c1 311.410
4 CHEMBL3452898 MMMKGEGGKRMNHL-UHFFFAOYSA-N Cn1ccc(C(=O)N=c2[nH]c(-c3ccc4c(c3)CCO4)cs2)cc1=O 353.403

Generating Morgan fingerprints of n_bits = 2048 and radius 3

In [7]:
from rdkit import DataStructs
import numpy as np

def fetch_fp(smile):
    fp = Chem.GetMorganFingerprintAsBitVect(MolFromSmiles(smile), 3, nBits = 2048)
    arr = np.zeros((0, ), dtype=np.int8)
    DataStructs.ConvertToNumpyArray(fp, arr)
    
    return(arr)
In [8]:
df['Fingerprint'] = df['Smiles'].map(fetch_fp)
df.head()
Out[8]:
chembl_id InChIKey Smiles MolWt Fingerprint
0 CHEMBL424856 KNZSNTNUOIRWRE-UHFFFAOYSA-N CC1(F)CC(F)(F)c2nn(-c3c(Cl)cc(C(F)(F)F)cc3Cl)cc21 389.126 [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, ...
1 CHEMBL3792491 NTQFMRZDTVXICG-UHFFFAOYSA-N CCN(CCCCCCNC(=O)C=Cc1ccc(O)c(OC)c1)Cc1ccccc1OC 440.584 [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, ...
2 CHEMBL3900710 IRTOHFGPHMCDRK-UHFFFAOYSA-N Cc1cc(N2C(=O)c3cc(-c4ccn(C)c(=O)c4)n(C4CC4)c3C... 500.986 [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, ...
3 CHEMBL35988 YBIWOQATDVKIPT-UHFFFAOYSA-N CC(O)c1cccc(C(C#N)C(=N)Sc2ccccc2N)c1 311.410 [0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, ...
4 CHEMBL3452898 MMMKGEGGKRMNHL-UHFFFAOYSA-N Cn1ccc(C(=O)N=c2[nH]c(-c3ccc4c(c3)CCO4)cs2)cc1=O 353.403 [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, ...

Let us see a distribution of MolWt

In [17]:
import matplotlib.pyplot as plt
%matplotlib inline

df.MolWt.plot.hist(alpha=0.5, figsize=(6, 5))
plt.xlabel('Average Molecular Weight Distribution')
plt.ylabel('Number of Molecules')

plt.show()

Let us split the data into train, validation and test sets

In [19]:
import sklearn.model_selection as model_selection
In [20]:
#Splitting full df into training and test set
df_train, df_test  = model_selection.train_test_split(df, train_size=0.8, test_size=0.2, random_state = 50)
df_train = df_train.reset_index(drop=True)
df_test = df_test.reset_index(drop=True)
In [21]:
#Splitting test set into validation and test set
df_test, df_val  = model_selection.train_test_split(df_test, train_size=0.8, test_size=0.2, random_state = 51)
df_test = df_test.reset_index(drop=True)
df_val = df_val.reset_index(drop=True)
In [22]:
df_train.info()
df_train.head()
<class 'pandas.core.frame.DataFrame'>
RangeIndex: 40000 entries, 0 to 39999
Data columns (total 5 columns):
 #   Column       Non-Null Count  Dtype  
---  ------       --------------  -----  
 0   chembl_id    40000 non-null  object 
 1   InChIKey     40000 non-null  object 
 2   Smiles       40000 non-null  object 
 3   MolWt        40000 non-null  float64
 4   Fingerprint  40000 non-null  object 
dtypes: float64(1), object(4)
memory usage: 1.5+ MB
Out[22]:
chembl_id InChIKey Smiles MolWt Fingerprint
0 CHEMBL1910118 OUZPFPWUZJUWBW-UHFFFAOYSA-N CCN(CC)C(=O)Cn1cc2c(-c3ccc(Cl)cc3Cl)c(CN)c(C)n... 435.355 [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, ...
1 CHEMBL3701718 OMHKRUKICMAVEQ-UHFFFAOYSA-N CC1(O)CC(N)(c2ccc(-c3nc4n(c3-c3ccccc3)COc3cccc... 441.506 [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, ...
2 CHEMBL1586469 IZGCNQQYKVLKJF-UHFFFAOYSA-N O=C(Nc1ccc(O)cc1)c1ccc2c(c1)C(=O)N(Cc1ccccc1)C2=O 372.380 [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, ...
3 CHEMBL15911 LXTWWFKWPYKGDZ-UHFFFAOYSA-N Nc1ccc2c(c1)Oc1ncccc1C(=O)N2 227.223 [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, ...
4 CHEMBL1977077 XCNVQBYFDVXDCR-UHFFFAOYSA-N COC(=NN=Cc1ccc(Cl)c(Cl)c1)c1ccncc1 308.168 [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, ...
In [23]:
df_test.info()
df_test.head()
<class 'pandas.core.frame.DataFrame'>
RangeIndex: 8000 entries, 0 to 7999
Data columns (total 5 columns):
 #   Column       Non-Null Count  Dtype  
---  ------       --------------  -----  
 0   chembl_id    8000 non-null   object 
 1   InChIKey     8000 non-null   object 
 2   Smiles       8000 non-null   object 
 3   MolWt        8000 non-null   float64
 4   Fingerprint  8000 non-null   object 
dtypes: float64(1), object(4)
memory usage: 312.6+ KB
Out[23]:
chembl_id InChIKey Smiles MolWt Fingerprint
0 CHEMBL3457366 GGROEQZNFUAMRC-UHFFFAOYSA-N Cc1nc2cc(=O)[nH]n2c(C)c1CC(=O)Nc1cnn(C(C)(C)C)c1 342.403 [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, ...
1 CHEMBL4082919 GIUFLJDLCSSQNX-UHFFFAOYSA-N Oc1c(C=C2C=CC=N2)oc2ccccc12 211.220 [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, ...
2 CHEMBL3823725 KYFDIOOCZMQBNF-UHFFFAOYSA-N O=C(N=c1[nH]c2ccc(Cl)cc2s1)Nc1ccc(C(=O)O)cc1 347.783 [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, ...
3 CHEMBL1205660 FBHBALAPMKYARW-UHFFFAOYSA-N NC(=O)Nc1ccc(NC(=O)N2CCC3C2C(=O)N3S(=O)(=O)O)cc1 369.359 [0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, ...
4 CHEMBL1371129 UWEYRJFJVCLAGH-UHFFFAOYSA-N FC1(F)C(F)(F)C(F)(F)C2(F)C(F)(F)C(F)(F)C(F)(F)... 462.074 [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, ...
In [24]:
df_val.info()
df_val.head()
<class 'pandas.core.frame.DataFrame'>
RangeIndex: 2000 entries, 0 to 1999
Data columns (total 5 columns):
 #   Column       Non-Null Count  Dtype  
---  ------       --------------  -----  
 0   chembl_id    2000 non-null   object 
 1   InChIKey     2000 non-null   object 
 2   Smiles       2000 non-null   object 
 3   MolWt        2000 non-null   float64
 4   Fingerprint  2000 non-null   object 
dtypes: float64(1), object(4)
memory usage: 78.2+ KB
Out[24]:
chembl_id InChIKey Smiles MolWt Fingerprint
0 CHEMBL1521162 SOWYDDIFKJDJQJ-UHFFFAOYSA-N Cc1ccc(C)c(CSCC(=O)NCc2cccs2)c1 305.468 [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, ...
1 CHEMBL2094512 VEGQQKXISPLKFE-UHFFFAOYSA-N O=c1c(-c2ccccc2)c(-c2ccccc2)[nH]c2ccccc12 297.357 [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, ...
2 CHEMBL1547923 NGGVSRYVZQZRCT-UHFFFAOYSA-N COc1ccc(CCN(C)CC=Cc2ccccc2[N+](=O)[O-])cc1OC 356.422 [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, ...
3 CHEMBL4292611 ANBMJCJTPBBGIB-UHFFFAOYSA-N O=S1(=O)C=C(SC(O)C(O)S)c2ccccc21 290.387 [0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, ...
4 CHEMBL3310133 SPXCQNAMVXGUIP-UHFFFAOYSA-N CC(=O)N=c1o[nH]c(-c2ccc(F)cc2)c1-c1ccncc1 297.289 [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, ...

Let us prepare the Xs and Ys

In [25]:
X_train = np.stack(df_train.Fingerprint)
X_test = np.stack(df_test.Fingerprint)
X_val = np.stack(df_val.Fingerprint)

Y_train = df_train['MolWt']
Y_test = df_test['MolWt']
Y_val = df_val['MolWt']

Y_train = Y_train.to_numpy()
Y_test = Y_test.to_numpy()
Y_val = Y_val.to_numpy()
In [26]:
print("Shape X_train: ", X_train.shape)
print("Shape Y_train: ", Y_train.shape)
print("Shape X_val: ", X_val.shape)
print("Shape Y_val: ", Y_val.shape)
print("Shape X_test: ", X_test.shape)
print ("Shape Y_test: ", Y_test.shape)
Shape X_train:  (40000, 2048)
Shape Y_train:  (40000,)
Shape X_val:  (2000, 2048)
Shape Y_val:  (2000,)
Shape X_test:  (8000, 2048)
Shape Y_test:  (8000,)

Now that we have our data preparations done, let us move to the modelling part.

There are 3 main reasons I like using Optuna

  1. Easy to install
  2. Easy to use
  3. Very verbose

We will cover all points.

1. Easy to install

One can simply use either use 'pip' or 'conda' to install Optuna. Optuna needs only handful of packages, so unnecessary cross-compatibility issues are avoided.

pip install optuna

or

conda install -c conda-forge optuna

2. Easy to use

Optuna is framework agnostic, that means it can run on practically any framework. Here's the best thing - the framework doesn't even need to be a machine learning framework!

We will do demo modelling on Tensorflow-Keras, Scikit-Learn's Random Forest, and XGBoost to see how it works.

Study and Trials

Practically only thing one must understand before the usage. In Optuna, one makes a study to optimize, for instance, a neural network model to fit your data. To optimize the hyperparameters of this study, trials with different hyperparameters are run. Optuna evaluates a trial once it is done training on its set of hyperparameters and selects the best trial. We can use this best trial's hyperparameters to then train the final model.

To show a very easy example describing study and trials, and optimization, I am taking this example (and modifying it slightly) posted on Optuna's main webpage:

In this example, we minimize x in this simple function: (x-2)^2. We suggest the values of x (as a float) between -5 and 5. The number of trials is set to 50, meaning, the whole process with different values of x will be repeated 100 times.

In [27]:
import optuna
In [29]:
def objective(trial):
    x = trial.suggest_float('x', -5, 5)
    return (x - 2) ** 2

study = optuna.create_study(direction = 'minimize', study_name = 'minimize_x_study')
study.optimize(objective, n_trials=100)
[I 2022-02-03 17:30:39,065] A new study created in memory with name: minimize_x_study
[I 2022-02-03 17:30:39,069] Trial 0 finished with value: 0.9108178695621746 and parameters: {'x': 1.045632214729471}. Best is trial 0 with value: 0.9108178695621746.
[I 2022-02-03 17:30:39,070] Trial 1 finished with value: 39.55697151495289 and parameters: {'x': -4.289433322243975}. Best is trial 0 with value: 0.9108178695621746.
[I 2022-02-03 17:30:39,071] Trial 2 finished with value: 3.1337800326197214 and parameters: {'x': 0.22975142067004573}. Best is trial 0 with value: 0.9108178695621746.
[I 2022-02-03 17:30:39,072] Trial 3 finished with value: 4.0969587798839635 and parameters: {'x': 4.024094558039215}. Best is trial 0 with value: 0.9108178695621746.
[I 2022-02-03 17:30:39,073] Trial 4 finished with value: 1.2131626435786214 and parameters: {'x': 0.8985633728722195}. Best is trial 0 with value: 0.9108178695621746.
[I 2022-02-03 17:30:39,074] Trial 5 finished with value: 0.06930082365025429 and parameters: {'x': 1.7367495039885883}. Best is trial 5 with value: 0.06930082365025429.
[I 2022-02-03 17:30:39,075] Trial 6 finished with value: 0.4012009797278609 and parameters: {'x': 1.3665957217322724}. Best is trial 5 with value: 0.06930082365025429.
[I 2022-02-03 17:30:39,076] Trial 7 finished with value: 0.21531595398293613 and parameters: {'x': 2.4640215016385945}. Best is trial 5 with value: 0.06930082365025429.
[I 2022-02-03 17:30:39,077] Trial 8 finished with value: 8.352179931237078 and parameters: {'x': -0.8900138289006643}. Best is trial 5 with value: 0.06930082365025429.
[I 2022-02-03 17:30:39,078] Trial 9 finished with value: 24.64057758101705 and parameters: {'x': -2.9639276365612996}. Best is trial 5 with value: 0.06930082365025429.
[I 2022-02-03 17:30:39,082] Trial 10 finished with value: 6.610843872804539 and parameters: {'x': 4.571156135438791}. Best is trial 5 with value: 0.06930082365025429.
[I 2022-02-03 17:30:39,085] Trial 11 finished with value: 1.167808969298569 and parameters: {'x': 3.080652103731154}. Best is trial 5 with value: 0.06930082365025429.
[I 2022-02-03 17:30:39,089] Trial 12 finished with value: 0.23811780553325998 and parameters: {'x': 2.4879731606689655}. Best is trial 5 with value: 0.06930082365025429.
[I 2022-02-03 17:30:39,092] Trial 13 finished with value: 0.3122370428412051 and parameters: {'x': 2.558781748844041}. Best is trial 5 with value: 0.06930082365025429.
[I 2022-02-03 17:30:39,095] Trial 14 finished with value: 11.547085781107892 and parameters: {'x': -1.3981003194590786}. Best is trial 5 with value: 0.06930082365025429.
[I 2022-02-03 17:30:39,099] Trial 15 finished with value: 0.004458701099944047 and parameters: {'x': 2.066773505973133}. Best is trial 15 with value: 0.004458701099944047.
[I 2022-02-03 17:30:39,102] Trial 16 finished with value: 8.288541515811726 and parameters: {'x': -0.8789827223885394}. Best is trial 15 with value: 0.004458701099944047.
[I 2022-02-03 17:30:39,106] Trial 17 finished with value: 2.5929541284411526 and parameters: {'x': 3.6102652354320863}. Best is trial 15 with value: 0.004458701099944047.
[I 2022-02-03 17:30:39,109] Trial 18 finished with value: 20.40847788303741 and parameters: {'x': -2.517574336193862}. Best is trial 15 with value: 0.004458701099944047.
[I 2022-02-03 17:30:39,113] Trial 19 finished with value: 0.11617214605732828 and parameters: {'x': 1.6591596472579453}. Best is trial 15 with value: 0.004458701099944047.
[I 2022-02-03 17:30:39,117] Trial 20 finished with value: 4.218511894235807 and parameters: {'x': -0.053901627205112046}. Best is trial 15 with value: 0.004458701099944047.
[I 2022-02-03 17:30:39,120] Trial 21 finished with value: 0.1916185079408783 and parameters: {'x': 1.5622574867106482}. Best is trial 15 with value: 0.004458701099944047.
[I 2022-02-03 17:30:39,124] Trial 22 finished with value: 0.01515838621131428 and parameters: {'x': 1.8768806018073745}. Best is trial 15 with value: 0.004458701099944047.
[I 2022-02-03 17:30:39,127] Trial 23 finished with value: 2.6721469014200747 and parameters: {'x': 0.3653297270030036}. Best is trial 15 with value: 0.004458701099944047.
[I 2022-02-03 17:30:39,131] Trial 24 finished with value: 8.19436875575276 and parameters: {'x': 4.862580785891074}. Best is trial 15 with value: 0.004458701099944047.
[I 2022-02-03 17:30:39,135] Trial 25 finished with value: 0.04574423430886892 and parameters: {'x': 2.2138790179257164}. Best is trial 15 with value: 0.004458701099944047.
[I 2022-02-03 17:30:39,138] Trial 26 finished with value: 2.655608178869724 and parameters: {'x': 3.6296036876706324}. Best is trial 15 with value: 0.004458701099944047.
[I 2022-02-03 17:30:39,142] Trial 27 finished with value: 0.8110549011138359 and parameters: {'x': 2.900585865486371}. Best is trial 15 with value: 0.004458701099944047.
[I 2022-02-03 17:30:39,146] Trial 28 finished with value: 0.006775211536097166 and parameters: {'x': 1.9176883268539804}. Best is trial 15 with value: 0.004458701099944047.
[I 2022-02-03 17:30:39,149] Trial 29 finished with value: 1.701921381721531 and parameters: {'x': 0.6954229107785423}. Best is trial 15 with value: 0.004458701099944047.
[I 2022-02-03 17:30:39,152] Trial 30 finished with value: 5.973108781164384 and parameters: {'x': -0.4439944314920982}. Best is trial 15 with value: 0.004458701099944047.
[I 2022-02-03 17:30:39,155] Trial 31 finished with value: 0.0018676007604320486 and parameters: {'x': 2.043215746672157}. Best is trial 31 with value: 0.0018676007604320486.
[I 2022-02-03 17:30:39,158] Trial 32 finished with value: 0.00020504899474743982 and parameters: {'x': 1.9856804680681441}. Best is trial 32 with value: 0.00020504899474743982.
[I 2022-02-03 17:30:39,161] Trial 33 finished with value: 1.4389540754099637 and parameters: {'x': 3.1995641189240214}. Best is trial 32 with value: 0.00020504899474743982.
[I 2022-02-03 17:30:39,164] Trial 34 finished with value: 0.5753703648072998 and parameters: {'x': 1.2414682835851227}. Best is trial 32 with value: 0.00020504899474743982.
[I 2022-02-03 17:30:39,167] Trial 35 finished with value: 4.351874589507826 and parameters: {'x': 4.086114711493073}. Best is trial 32 with value: 0.00020504899474743982.
[I 2022-02-03 17:30:39,170] Trial 36 finished with value: 1.2715580826950736 and parameters: {'x': 0.8723661575249377}. Best is trial 32 with value: 0.00020504899474743982.
[I 2022-02-03 17:30:39,174] Trial 37 finished with value: 0.02152907409581088 and parameters: {'x': 1.8532721086643344}. Best is trial 32 with value: 0.00020504899474743982.
[I 2022-02-03 17:30:39,177] Trial 38 finished with value: 41.32704168425618 and parameters: {'x': -4.428611178493857}. Best is trial 32 with value: 0.00020504899474743982.
[I 2022-02-03 17:30:39,180] Trial 39 finished with value: 2.336371398090483 and parameters: {'x': 0.47148065171209463}. Best is trial 32 with value: 0.00020504899474743982.
[I 2022-02-03 17:30:39,183] Trial 40 finished with value: 0.011032167280930281 and parameters: {'x': 2.105034124364086}. Best is trial 32 with value: 0.00020504899474743982.
[I 2022-02-03 17:30:39,186] Trial 41 finished with value: 0.025751722129564013 and parameters: {'x': 2.1604734312263685}. Best is trial 32 with value: 0.00020504899474743982.
[I 2022-02-03 17:30:39,189] Trial 42 finished with value: 0.7318796258085658 and parameters: {'x': 1.1445003648109686}. Best is trial 32 with value: 0.00020504899474743982.
[I 2022-02-03 17:30:39,192] Trial 43 finished with value: 0.8180487796528402 and parameters: {'x': 2.9044604909297256}. Best is trial 32 with value: 0.00020504899474743982.
[I 2022-02-03 17:30:39,195] Trial 44 finished with value: 2.384170378980506 and parameters: {'x': 3.5440758980634683}. Best is trial 32 with value: 0.00020504899474743982.
[I 2022-02-03 17:30:39,199] Trial 45 finished with value: 4.31931587902329 and parameters: {'x': 4.0782963886374075}. Best is trial 32 with value: 0.00020504899474743982.
[I 2022-02-03 17:30:39,202] Trial 46 finished with value: 0.45542295534188787 and parameters: {'x': 2.67485032069481}. Best is trial 32 with value: 0.00020504899474743982.
[I 2022-02-03 17:30:39,205] Trial 47 finished with value: 0.09735169760265308 and parameters: {'x': 2.312012335657828}. Best is trial 32 with value: 0.00020504899474743982.
[I 2022-02-03 17:30:39,208] Trial 48 finished with value: 0.6862064865784465 and parameters: {'x': 1.1716241875945155}. Best is trial 32 with value: 0.00020504899474743982.
[I 2022-02-03 17:30:39,211] Trial 49 finished with value: 4.106391156036289 and parameters: {'x': -0.026423242078586373}. Best is trial 32 with value: 0.00020504899474743982.
[I 2022-02-03 17:30:39,215] Trial 50 finished with value: 48.681255281433515 and parameters: {'x': -4.977195373603459}. Best is trial 32 with value: 0.00020504899474743982.
[I 2022-02-03 17:30:39,218] Trial 51 finished with value: 0.018645841427809423 and parameters: {'x': 1.8634502236259267}. Best is trial 32 with value: 0.00020504899474743982.
[I 2022-02-03 17:30:39,221] Trial 52 finished with value: 0.009269649536348689 and parameters: {'x': 2.0962790191908325}. Best is trial 32 with value: 0.00020504899474743982.
[I 2022-02-03 17:30:39,225] Trial 53 finished with value: 1.534359596401101 and parameters: {'x': 3.238692696515605}. Best is trial 32 with value: 0.00020504899474743982.
[I 2022-02-03 17:30:39,228] Trial 54 finished with value: 0.37529334643960965 and parameters: {'x': 1.3873880947617736}. Best is trial 32 with value: 0.00020504899474743982.
[I 2022-02-03 17:30:39,231] Trial 55 finished with value: 0.017784223077757618 and parameters: {'x': 2.133357501017969}. Best is trial 32 with value: 0.00020504899474743982.
[I 2022-02-03 17:30:39,234] Trial 56 finished with value: 0.33091453205610916 and parameters: {'x': 2.575251711910629}. Best is trial 32 with value: 0.00020504899474743982.
[I 2022-02-03 17:30:39,237] Trial 57 finished with value: 0.1437866663110277 and parameters: {'x': 1.620807876781403}. Best is trial 32 with value: 0.00020504899474743982.
[I 2022-02-03 17:30:39,241] Trial 58 finished with value: 15.030168917655432 and parameters: {'x': -1.8768761803358425}. Best is trial 32 with value: 0.00020504899474743982.
[I 2022-02-03 17:30:39,244] Trial 59 finished with value: 1.5972461813623473 and parameters: {'x': 0.7361779471134604}. Best is trial 32 with value: 0.00020504899474743982.
[I 2022-02-03 17:30:39,247] Trial 60 finished with value: 0.6197896243145568 and parameters: {'x': 2.78726718737323}. Best is trial 32 with value: 0.00020504899474743982.
[I 2022-02-03 17:30:39,250] Trial 61 finished with value: 0.0025272231843381106 and parameters: {'x': 1.9497285052506084}. Best is trial 32 with value: 0.00020504899474743982.
[I 2022-02-03 17:30:39,253] Trial 62 finished with value: 0.00028264886553565604 and parameters: {'x': 2.0168121642133205}. Best is trial 32 with value: 0.00020504899474743982.
[I 2022-02-03 17:30:39,256] Trial 63 finished with value: 0.20539815210173015 and parameters: {'x': 1.5467912709338774}. Best is trial 32 with value: 0.00020504899474743982.
[I 2022-02-03 17:30:39,260] Trial 64 finished with value: 0.24708715669952516 and parameters: {'x': 2.497078622251576}. Best is trial 32 with value: 0.00020504899474743982.
[I 2022-02-03 17:30:39,263] Trial 65 finished with value: 0.006012535952778148 and parameters: {'x': 1.9224594560711743}. Best is trial 32 with value: 0.00020504899474743982.
[I 2022-02-03 17:30:39,266] Trial 66 finished with value: 2.037330447122795 and parameters: {'x': 3.427350849343915}. Best is trial 32 with value: 0.00020504899474743982.
[I 2022-02-03 17:30:39,269] Trial 67 finished with value: 5.748204778822988 and parameters: {'x': -0.39754140294239493}. Best is trial 32 with value: 0.00020504899474743982.
[I 2022-02-03 17:30:39,273] Trial 68 finished with value: 1.021608762937964 and parameters: {'x': 0.9892533636276776}. Best is trial 32 with value: 0.00020504899474743982.
[I 2022-02-03 17:30:39,276] Trial 69 finished with value: 0.026031423835400466 and parameters: {'x': 1.8386574332812309}. Best is trial 32 with value: 0.00020504899474743982.
[I 2022-02-03 17:30:39,279] Trial 70 finished with value: 3.5427383127206804 and parameters: {'x': 3.8822163299474055}. Best is trial 32 with value: 0.00020504899474743982.
[I 2022-02-03 17:30:39,282] Trial 71 finished with value: 0.0010073653525717768 and parameters: {'x': 2.031739019401547}. Best is trial 32 with value: 0.00020504899474743982.
[I 2022-02-03 17:30:39,286] Trial 72 finished with value: 0.34470320072185573 and parameters: {'x': 1.4128857004621198}. Best is trial 32 with value: 0.00020504899474743982.
[I 2022-02-03 17:30:39,289] Trial 73 finished with value: 0.9934678755759706 and parameters: {'x': 2.9967285867155464}. Best is trial 32 with value: 0.00020504899474743982.
[I 2022-02-03 17:30:39,292] Trial 74 finished with value: 0.11877279349528397 and parameters: {'x': 2.3446342894943624}. Best is trial 32 with value: 0.00020504899474743982.
[I 2022-02-03 17:30:39,295] Trial 75 finished with value: 2.036362883955774 and parameters: {'x': 0.5729881276051787}. Best is trial 32 with value: 0.00020504899474743982.
[I 2022-02-03 17:30:39,298] Trial 76 finished with value: 3.0113144949582926 and parameters: {'x': 0.2646860529119543}. Best is trial 32 with value: 0.00020504899474743982.
[I 2022-02-03 17:30:39,302] Trial 77 finished with value: 0.00017118486971402487 and parameters: {'x': 2.013083763591338}. Best is trial 77 with value: 0.00017118486971402487.
[I 2022-02-03 17:30:39,305] Trial 78 finished with value: 0.12880879351970892 and parameters: {'x': 1.6411005802182053}. Best is trial 77 with value: 0.00017118486971402487.
[I 2022-02-03 17:30:39,308] Trial 79 finished with value: 0.40022989405650855 and parameters: {'x': 2.632637253136826}. Best is trial 77 with value: 0.00017118486971402487.
[I 2022-02-03 17:30:39,312] Trial 80 finished with value: 0.9411687665491552 and parameters: {'x': 1.0298614704336524}. Best is trial 77 with value: 0.00017118486971402487.
[I 2022-02-03 17:30:39,315] Trial 81 finished with value: 0.0004672763104167905 and parameters: {'x': 1.9783834250997807}. Best is trial 77 with value: 0.00017118486971402487.
[I 2022-02-03 17:30:39,319] Trial 82 finished with value: 2.116257428664426e-07 and parameters: {'x': 1.9995399720194744}. Best is trial 82 with value: 2.116257428664426e-07.
[I 2022-02-03 17:30:39,322] Trial 83 finished with value: 0.07499502290372632 and parameters: {'x': 2.2738521917088237}. Best is trial 82 with value: 2.116257428664426e-07.
[I 2022-02-03 17:30:39,325] Trial 84 finished with value: 0.43519382358857633 and parameters: {'x': 1.3403077811671746}. Best is trial 82 with value: 2.116257428664426e-07.
[I 2022-02-03 17:30:39,329] Trial 85 finished with value: 1.6223259378830002 and parameters: {'x': 3.2737055930955945}. Best is trial 82 with value: 2.116257428664426e-07.
[I 2022-02-03 17:30:39,332] Trial 86 finished with value: 0.6315443195656962 and parameters: {'x': 2.794697627758946}. Best is trial 82 with value: 2.116257428664426e-07.
[I 2022-02-03 17:30:39,335] Trial 87 finished with value: 0.0019290556635620229 and parameters: {'x': 2.043921016194551}. Best is trial 82 with value: 2.116257428664426e-07.
[I 2022-02-03 17:30:39,338] Trial 88 finished with value: 0.06531306931317488 and parameters: {'x': 1.7444357824084622}. Best is trial 82 with value: 2.116257428664426e-07.
[I 2022-02-03 17:30:39,342] Trial 89 finished with value: 0.10385703453465352 and parameters: {'x': 2.3222685751584438}. Best is trial 82 with value: 2.116257428664426e-07.
[I 2022-02-03 17:30:39,346] Trial 90 finished with value: 1.1476541825037925 and parameters: {'x': 3.071286228093964}. Best is trial 82 with value: 2.116257428664426e-07.
[I 2022-02-03 17:30:39,349] Trial 91 finished with value: 6.560707063935156e-05 and parameters: {'x': 1.9919001808267498}. Best is trial 82 with value: 2.116257428664426e-07.
[I 2022-02-03 17:30:39,353] Trial 92 finished with value: 0.00029300833371463523 and parameters: {'x': 1.9828825138027035}. Best is trial 82 with value: 2.116257428664426e-07.
[I 2022-02-03 17:30:39,356] Trial 93 finished with value: 0.2474140052647107 and parameters: {'x': 2.4974072830837026}. Best is trial 82 with value: 2.116257428664426e-07.
[I 2022-02-03 17:30:39,360] Trial 94 finished with value: 0.23233673824714104 and parameters: {'x': 1.5179867862318077}. Best is trial 82 with value: 2.116257428664426e-07.
[I 2022-02-03 17:30:39,363] Trial 95 finished with value: 0.0008365995533424821 and parameters: {'x': 2.0289240307243386}. Best is trial 82 with value: 2.116257428664426e-07.
[I 2022-02-03 17:30:39,366] Trial 96 finished with value: 0.5732761237239836 and parameters: {'x': 1.2428499991917166}. Best is trial 82 with value: 2.116257428664426e-07.
[I 2022-02-03 17:30:39,370] Trial 97 finished with value: 29.093803323523407 and parameters: {'x': -3.393867195577159}. Best is trial 82 with value: 2.116257428664426e-07.
[I 2022-02-03 17:30:39,373] Trial 98 finished with value: 1.4029344346511763 and parameters: {'x': 0.8155446675154119}. Best is trial 82 with value: 2.116257428664426e-07.
[I 2022-02-03 17:30:39,376] Trial 99 finished with value: 0.0765794337738179 and parameters: {'x': 1.7232701068301115}. Best is trial 82 with value: 2.116257428664426e-07.
In [30]:
study.best_params
Out[30]:
{'x': 1.9995399720194744}

We can see the best trial found was trial 82 with the value 2 x 10^-7 returned, when x = 1.99. This example also shows that Optuna can be used to practically tune any function as long as a value is returned which can determine the direction in which the parameter(s) have to be optimized.

Back to ML algorithms and our problem statement

I will begin with Tensorflow-Keras since I use them most of the time.

In [31]:
import tensorflow as tf
from tensorflow.keras.models import Sequential 
from tensorflow.keras.layers import Dense, Activation, Dropout
from tensorflow.keras.utils import plot_model
2022-02-03 17:43:19.688986: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.10.1
In [ ]:
#TF configuration
print(tf.__version__)

gpus = tf.config.list_physical_devices('GPU')
if gpus:
  # Restrict TensorFlow to only use the first GPU
    try:
        tf.config.experimental.set_visible_devices(gpus[0], 'GPU')
        logical_gpus = tf.config.experimental.list_logical_devices('GPU')
        print(len(gpus), "Physical GPUs,", len(logical_gpus), "Logical GPU")
    except RuntimeError as e:
    # Visible devices must be set before GPUs have been initialized
        print(e)
In [35]:
def objective(trial):
    
    # We optimize the numbers of layers and their units.
    n_layers = trial.suggest_int("n_layers", 1, 5)
    
    #Optimizing number of epochs
    n_epochs = trial.suggest_int("n_epochs", 1, 10)
    
    model = tf.keras.Sequential()
    
    for i in range(n_layers):
        num_hidden = trial.suggest_int("n_units_l{}".format(i), 10, 2000, log=True)
        model.add(Dense(num_hidden, activation="relu"))
    
    
    model.add(Dropout(0.2))
    model.add(Dense(1, activation='linear'))
        
    #Optimizing the learning rate
    lr = trial.suggest_float('lr', 1e-5, 1e-1, log=True)
    
    model.compile(loss="mean_squared_error", optimizer= tf.keras.optimizers.Adam(lr=lr), metrics=["mean_absolute_error"])
    
    #Optimizing batch size
    n_batchsize = trial.suggest_int("n_batchsize", 128, 512)
    
    model.fit(
    X_train,
    Y_train,
    #validation_data=(X_val, Y_val),
    shuffle=True,
    batch_size=n_batchsize,
    epochs=n_epochs,
    verbose=False)
        
    eval_loss, eval_accuracy = model.evaluate(X_val, Y_val, verbose=0)
        
    return eval_loss
In [37]:
study = optuna.create_study(direction='minimize', study_name = 'tf-keras_model_optimization')
study.optimize(objective, n_trials=500)
[I 2022-02-03 17:50:42,462] A new study created in memory with name: tf-keras_model_optimization
2022-02-03 17:50:56.489663: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:116] None of the MLIR optimization passes are enabled (registered 2)
2022-02-03 17:50:56.989504: I tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 2900000000 Hz
2022-02-03 17:51:00.776413: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.10
[I 2022-02-03 17:51:34,104] Trial 0 finished with value: 1598.263427734375 and parameters: {'n_layers': 3, 'n_epochs': 9, 'n_units_l0': 165, 'n_units_l1': 28, 'n_units_l2': 630, 'lr': 0.001960764970029559, 'n_batchsize': 149}. Best is trial 0 with value: 1598.263427734375.
[I 2022-02-03 17:51:35,171] Trial 1 finished with value: 1965.693603515625 and parameters: {'n_layers': 4, 'n_epochs': 2, 'n_units_l0': 126, 'n_units_l1': 12, 'n_units_l2': 17, 'n_units_l3': 143, 'lr': 0.0011779294014049734, 'n_batchsize': 323}. Best is trial 0 with value: 1598.263427734375.
[I 2022-02-03 17:51:37,614] Trial 2 finished with value: 1630.947021484375 and parameters: {'n_layers': 3, 'n_epochs': 6, 'n_units_l0': 220, 'n_units_l1': 256, 'n_units_l2': 50, 'lr': 0.010219477659906159, 'n_batchsize': 244}. Best is trial 0 with value: 1598.263427734375.
[I 2022-02-03 17:51:40,522] Trial 3 finished with value: 1738.7030029296875 and parameters: {'n_layers': 2, 'n_epochs': 8, 'n_units_l0': 10, 'n_units_l1': 29, 'lr': 0.006626563322648165, 'n_batchsize': 238}. Best is trial 0 with value: 1598.263427734375.
[I 2022-02-03 17:51:42,673] Trial 4 finished with value: 158643.921875 and parameters: {'n_layers': 1, 'n_epochs': 10, 'n_units_l0': 263, 'lr': 1.0017861622584726e-05, 'n_batchsize': 380}. Best is trial 0 with value: 1598.263427734375.
[I 2022-02-03 17:51:45,448] Trial 5 finished with value: 1472.562744140625 and parameters: {'n_layers': 5, 'n_epochs': 9, 'n_units_l0': 1603, 'n_units_l1': 301, 'n_units_l2': 10, 'n_units_l3': 697, 'n_units_l4': 78, 'lr': 0.00041075642022433636, 'n_batchsize': 390}. Best is trial 5 with value: 1472.562744140625.
[I 2022-02-03 17:51:47,728] Trial 6 finished with value: 1728.1363525390625 and parameters: {'n_layers': 1, 'n_epochs': 9, 'n_units_l0': 668, 'lr': 0.0008637256234087589, 'n_batchsize': 326}. Best is trial 5 with value: 1472.562744140625.
[I 2022-02-03 17:51:49,836] Trial 7 finished with value: 130512.9765625 and parameters: {'n_layers': 1, 'n_epochs': 3, 'n_units_l0': 227, 'lr': 3.9625641542143066e-05, 'n_batchsize': 131}. Best is trial 5 with value: 1472.562744140625.
[I 2022-02-03 17:51:51,100] Trial 8 finished with value: 2171.565185546875 and parameters: {'n_layers': 1, 'n_epochs': 4, 'n_units_l0': 253, 'lr': 0.0015819967562721471, 'n_batchsize': 362}. Best is trial 5 with value: 1472.562744140625.
[I 2022-02-03 17:51:51,872] Trial 9 finished with value: 1838.8133544921875 and parameters: {'n_layers': 4, 'n_epochs': 1, 'n_units_l0': 50, 'n_units_l1': 17, 'n_units_l2': 12, 'n_units_l3': 32, 'lr': 0.03068691606278419, 'n_batchsize': 397}. Best is trial 5 with value: 1472.562744140625.
[I 2022-02-03 17:51:53,939] Trial 10 finished with value: 1605.7333984375 and parameters: {'n_layers': 5, 'n_epochs': 6, 'n_units_l0': 1584, 'n_units_l1': 1493, 'n_units_l2': 201, 'n_units_l3': 1928, 'n_units_l4': 79, 'lr': 0.00013024246384351322, 'n_batchsize': 512}. Best is trial 5 with value: 1472.562744140625.
[I 2022-02-03 17:51:55,997] Trial 11 finished with value: 1464.0380859375 and parameters: {'n_layers': 3, 'n_epochs': 8, 'n_units_l0': 1943, 'n_units_l1': 166, 'n_units_l2': 1359, 'lr': 0.0001887637089500753, 'n_batchsize': 458}. Best is trial 11 with value: 1464.0380859375.
[I 2022-02-03 17:51:58,140] Trial 12 finished with value: 1859.5111083984375 and parameters: {'n_layers': 5, 'n_epochs': 7, 'n_units_l0': 1776, 'n_units_l1': 218, 'n_units_l2': 1164, 'n_units_l3': 983, 'n_units_l4': 19, 'lr': 0.00014622332992093344, 'n_batchsize': 474}. Best is trial 11 with value: 1464.0380859375.
[I 2022-02-03 17:52:00,752] Trial 13 finished with value: 1412.7125244140625 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 837, 'n_units_l1': 541, 'n_units_l2': 275, 'n_units_l3': 312, 'lr': 0.00021405779316001597, 'n_batchsize': 441}. Best is trial 13 with value: 1412.7125244140625.
[I 2022-02-03 17:52:03,196] Trial 14 finished with value: 1655.6363525390625 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 738, 'n_units_l1': 1489, 'n_units_l2': 322, 'n_units_l3': 153, 'lr': 6.779633246008706e-05, 'n_batchsize': 462}. Best is trial 13 with value: 1412.7125244140625.
[I 2022-02-03 17:52:04,938] Trial 15 finished with value: 1785.067626953125 and parameters: {'n_layers': 2, 'n_epochs': 7, 'n_units_l0': 646, 'n_units_l1': 76, 'lr': 0.00032169469686322644, 'n_batchsize': 439}. Best is trial 13 with value: 1412.7125244140625.
[I 2022-02-03 17:52:06,948] Trial 16 finished with value: 2831.40673828125 and parameters: {'n_layers': 3, 'n_epochs': 8, 'n_units_l0': 842, 'n_units_l1': 626, 'n_units_l2': 1649, 'lr': 1.4486221544902307e-05, 'n_batchsize': 502}. Best is trial 13 with value: 1412.7125244140625.
[I 2022-02-03 17:52:08,480] Trial 17 finished with value: 155008.578125 and parameters: {'n_layers': 4, 'n_epochs': 5, 'n_units_l0': 63, 'n_units_l1': 80, 'n_units_l2': 76, 'n_units_l3': 10, 'lr': 3.652534978258038e-05, 'n_batchsize': 430}. Best is trial 13 with value: 1412.7125244140625.
[I 2022-02-03 17:52:11,588] Trial 18 finished with value: 1483.884033203125 and parameters: {'n_layers': 2, 'n_epochs': 10, 'n_units_l0': 417, 'n_units_l1': 609, 'lr': 0.0003481894019938587, 'n_batchsize': 267}. Best is trial 13 with value: 1412.7125244140625.
[I 2022-02-03 17:52:13,794] Trial 19 finished with value: 1461.676513671875 and parameters: {'n_layers': 4, 'n_epochs': 8, 'n_units_l0': 1127, 'n_units_l1': 107, 'n_units_l2': 505, 'n_units_l3': 345, 'lr': 0.004816234633425323, 'n_batchsize': 424}. Best is trial 13 with value: 1412.7125244140625.
[I 2022-02-03 17:52:15,704] Trial 20 finished with value: 2732.569091796875 and parameters: {'n_layers': 4, 'n_epochs': 7, 'n_units_l0': 94, 'n_units_l1': 79, 'n_units_l2': 396, 'n_units_l3': 348, 'lr': 0.08421865640199931, 'n_batchsize': 417}. Best is trial 13 with value: 1412.7125244140625.
[I 2022-02-03 17:52:17,735] Trial 21 finished with value: 1445.771240234375 and parameters: {'n_layers': 3, 'n_epochs': 8, 'n_units_l0': 1204, 'n_units_l1': 141, 'n_units_l2': 754, 'lr': 0.005506777920224659, 'n_batchsize': 462}. Best is trial 13 with value: 1412.7125244140625.
[I 2022-02-03 17:52:19,968] Trial 22 finished with value: 1607.554443359375 and parameters: {'n_layers': 3, 'n_epochs': 9, 'n_units_l0': 1071, 'n_units_l1': 588, 'n_units_l2': 648, 'lr': 0.0038953765489613836, 'n_batchsize': 484}. Best is trial 13 with value: 1412.7125244140625.
[I 2022-02-03 17:52:22,868] Trial 23 finished with value: 1683.8414306640625 and parameters: {'n_layers': 4, 'n_epochs': 8, 'n_units_l0': 462, 'n_units_l1': 50, 'n_units_l2': 134, 'n_units_l3': 270, 'lr': 0.01167525067952364, 'n_batchsize': 356}. Best is trial 13 with value: 1412.7125244140625.
[I 2022-02-03 17:52:25,771] Trial 24 finished with value: 1840.88525390625 and parameters: {'n_layers': 5, 'n_epochs': 10, 'n_units_l0': 1169, 'n_units_l1': 123, 'n_units_l2': 671, 'n_units_l3': 64, 'n_units_l4': 1572, 'lr': 0.02544532146400356, 'n_batchsize': 416}. Best is trial 13 with value: 1412.7125244140625.
[I 2022-02-03 17:52:27,845] Trial 25 finished with value: 1533.4312744140625 and parameters: {'n_layers': 4, 'n_epochs': 7, 'n_units_l0': 428, 'n_units_l1': 348, 'n_units_l2': 320, 'n_units_l3': 448, 'lr': 0.004113112547916256, 'n_batchsize': 447}. Best is trial 13 with value: 1412.7125244140625.
[I 2022-02-03 17:52:30,183] Trial 26 finished with value: 1422.6383056640625 and parameters: {'n_layers': 2, 'n_epochs': 9, 'n_units_l0': 1120, 'n_units_l1': 140, 'lr': 0.0006243047472944542, 'n_batchsize': 358}. Best is trial 13 with value: 1412.7125244140625.
[I 2022-02-03 17:52:32,890] Trial 27 finished with value: 1396.0767822265625 and parameters: {'n_layers': 2, 'n_epochs': 9, 'n_units_l0': 442, 'n_units_l1': 493, 'lr': 0.000657495215180861, 'n_batchsize': 283}. Best is trial 27 with value: 1396.0767822265625.
[I 2022-02-03 17:52:35,787] Trial 28 finished with value: 1442.3131103515625 and parameters: {'n_layers': 2, 'n_epochs': 9, 'n_units_l0': 361, 'n_units_l1': 1038, 'lr': 0.0007280516159085711, 'n_batchsize': 271}. Best is trial 27 with value: 1396.0767822265625.
[I 2022-02-03 17:52:39,047] Trial 29 finished with value: 1584.326904296875 and parameters: {'n_layers': 2, 'n_epochs': 9, 'n_units_l0': 17, 'n_units_l1': 405, 'lr': 0.0026769506839973167, 'n_batchsize': 204}. Best is trial 27 with value: 1396.0767822265625.
[I 2022-02-03 17:52:42,019] Trial 30 finished with value: 1496.7247314453125 and parameters: {'n_layers': 2, 'n_epochs': 10, 'n_units_l0': 546, 'n_units_l1': 997, 'lr': 0.0005394898855978837, 'n_batchsize': 294}. Best is trial 27 with value: 1396.0767822265625.
[I 2022-02-03 17:52:45,960] Trial 31 finished with value: 1405.861083984375 and parameters: {'n_layers': 2, 'n_epochs': 9, 'n_units_l0': 335, 'n_units_l1': 874, 'lr': 0.0007324722960881506, 'n_batchsize': 182}. Best is trial 27 with value: 1396.0767822265625.
[I 2022-02-03 17:52:49,764] Trial 32 finished with value: 1805.86279296875 and parameters: {'n_layers': 2, 'n_epochs': 10, 'n_units_l0': 137, 'n_units_l1': 861, 'lr': 0.0017955567909520675, 'n_batchsize': 199}. Best is trial 27 with value: 1396.0767822265625.
[I 2022-02-03 17:52:53,413] Trial 33 finished with value: 1578.362060546875 and parameters: {'n_layers': 2, 'n_epochs': 9, 'n_units_l0': 306, 'n_units_l1': 473, 'lr': 0.00024213870594614977, 'n_batchsize': 192}. Best is trial 27 with value: 1396.0767822265625.
[I 2022-02-03 17:52:56,119] Trial 34 finished with value: 1628.9632568359375 and parameters: {'n_layers': 1, 'n_epochs': 6, 'n_units_l0': 843, 'lr': 0.0010956355050476699, 'n_batchsize': 175}. Best is trial 27 with value: 1396.0767822265625.
[I 2022-02-03 17:52:58,903] Trial 35 finished with value: 2005.2801513671875 and parameters: {'n_layers': 2, 'n_epochs': 10, 'n_units_l0': 177, 'n_units_l1': 1732, 'lr': 7.968881044135979e-05, 'n_batchsize': 328}. Best is trial 27 with value: 1396.0767822265625.
[I 2022-02-03 17:53:01,806] Trial 36 finished with value: 1470.7669677734375 and parameters: {'n_layers': 3, 'n_epochs': 9, 'n_units_l0': 329, 'n_units_l1': 782, 'n_units_l2': 32, 'lr': 0.0006891708290228029, 'n_batchsize': 302}. Best is trial 27 with value: 1396.0767822265625.
[I 2022-02-03 17:53:03,965] Trial 37 finished with value: 1440.5435791015625 and parameters: {'n_layers': 3, 'n_epochs': 5, 'n_units_l0': 585, 'n_units_l1': 202, 'n_units_l2': 120, 'lr': 0.0004894068867129669, 'n_batchsize': 237}. Best is trial 27 with value: 1396.0767822265625.
[I 2022-02-03 17:53:06,192] Trial 38 finished with value: 1956.91650390625 and parameters: {'n_layers': 1, 'n_epochs': 9, 'n_units_l0': 174, 'lr': 0.0011288106392127102, 'n_batchsize': 349}. Best is trial 27 with value: 1396.0767822265625.
[I 2022-02-03 17:53:09,629] Trial 39 finished with value: 1384.86474609375 and parameters: {'n_layers': 2, 'n_epochs': 8, 'n_units_l0': 975, 'n_units_l1': 420, 'lr': 0.0026101743792820717, 'n_batchsize': 166}. Best is trial 39 with value: 1384.86474609375.
[I 2022-02-03 17:53:13,165] Trial 40 finished with value: 1686.38720703125 and parameters: {'n_layers': 1, 'n_epochs': 8, 'n_units_l0': 104, 'lr': 0.0026340633148626744, 'n_batchsize': 164}. Best is trial 39 with value: 1384.86474609375.
[I 2022-02-03 17:53:17,028] Trial 41 finished with value: 1314.6368408203125 and parameters: {'n_layers': 2, 'n_epochs': 9, 'n_units_l0': 940, 'n_units_l1': 445, 'lr': 0.001570269834522352, 'n_batchsize': 220}. Best is trial 41 with value: 1314.6368408203125.
[I 2022-02-03 17:53:20,520] Trial 42 finished with value: 1393.4613037109375 and parameters: {'n_layers': 2, 'n_epochs': 10, 'n_units_l0': 824, 'n_units_l1': 452, 'lr': 0.0015099745879868483, 'n_batchsize': 225}. Best is trial 41 with value: 1314.6368408203125.
[I 2022-02-03 17:53:23,796] Trial 43 finished with value: 1390.6048583984375 and parameters: {'n_layers': 2, 'n_epochs': 9, 'n_units_l0': 547, 'n_units_l1': 284, 'lr': 0.0015539647043264124, 'n_batchsize': 216}. Best is trial 41 with value: 1314.6368408203125.
[I 2022-02-03 17:53:26,869] Trial 44 finished with value: 1436.4366455078125 and parameters: {'n_layers': 2, 'n_epochs': 8, 'n_units_l0': 1508, 'n_units_l1': 299, 'lr': 0.0016038904240636288, 'n_batchsize': 222}. Best is trial 41 with value: 1314.6368408203125.
[I 2022-02-03 17:53:27,579] Trial 45 finished with value: 1841.4228515625 and parameters: {'n_layers': 1, 'n_epochs': 1, 'n_units_l0': 546, 'lr': 0.01092613508163635, 'n_batchsize': 258}. Best is trial 41 with value: 1314.6368408203125.
[I 2022-02-03 17:53:31,226] Trial 46 finished with value: 1360.4678955078125 and parameters: {'n_layers': 2, 'n_epochs': 7, 'n_units_l0': 1435, 'n_units_l1': 380, 'lr': 0.0018478502682065794, 'n_batchsize': 150}. Best is trial 41 with value: 1314.6368408203125.
[I 2022-02-03 17:53:35,542] Trial 47 finished with value: 1732.6505126953125 and parameters: {'n_layers': 3, 'n_epochs': 7, 'n_units_l0': 1497, 'n_units_l1': 235, 'n_units_l2': 27, 'lr': 0.0022310934321342476, 'n_batchsize': 131}. Best is trial 41 with value: 1314.6368408203125.
[I 2022-02-03 17:53:38,551] Trial 48 finished with value: 1530.9893798828125 and parameters: {'n_layers': 1, 'n_epochs': 6, 'n_units_l0': 884, 'lr': 0.003402971328264385, 'n_batchsize': 149}. Best is trial 41 with value: 1314.6368408203125.
[I 2022-02-03 17:53:40,068] Trial 49 finished with value: 1536.7781982421875 and parameters: {'n_layers': 2, 'n_epochs': 3, 'n_units_l0': 1981, 'n_units_l1': 375, 'lr': 0.007675142475322442, 'n_batchsize': 222}. Best is trial 41 with value: 1314.6368408203125.
[I 2022-02-03 17:53:43,669] Trial 50 finished with value: 1282.2197265625 and parameters: {'n_layers': 2, 'n_epochs': 7, 'n_units_l0': 1347, 'n_units_l1': 331, 'lr': 0.00109020597846138, 'n_batchsize': 151}. Best is trial 50 with value: 1282.2197265625.
[I 2022-02-03 17:53:47,119] Trial 51 finished with value: 1387.126953125 and parameters: {'n_layers': 2, 'n_epochs': 7, 'n_units_l0': 1332, 'n_units_l1': 289, 'lr': 0.001637445022952, 'n_batchsize': 155}. Best is trial 50 with value: 1282.2197265625.
[I 2022-02-03 17:53:50,612] Trial 52 finished with value: 1300.319580078125 and parameters: {'n_layers': 2, 'n_epochs': 7, 'n_units_l0': 1366, 'n_units_l1': 279, 'lr': 0.0014937424629818416, 'n_batchsize': 151}. Best is trial 50 with value: 1282.2197265625.
[I 2022-02-03 17:53:54,183] Trial 53 finished with value: 1337.1348876953125 and parameters: {'n_layers': 2, 'n_epochs': 7, 'n_units_l0': 1409, 'n_units_l1': 181, 'lr': 0.0010705535179519464, 'n_batchsize': 150}. Best is trial 50 with value: 1282.2197265625.
[I 2022-02-03 17:53:57,764] Trial 54 finished with value: 1283.91455078125 and parameters: {'n_layers': 2, 'n_epochs': 6, 'n_units_l0': 980, 'n_units_l1': 183, 'lr': 0.0009329022773739882, 'n_batchsize': 129}. Best is trial 50 with value: 1282.2197265625.
[I 2022-02-03 17:54:00,675] Trial 55 finished with value: 1732.3924560546875 and parameters: {'n_layers': 1, 'n_epochs': 5, 'n_units_l0': 1622, 'lr': 0.0004059711850873653, 'n_batchsize': 131}. Best is trial 50 with value: 1282.2197265625.
[I 2022-02-03 17:54:04,020] Trial 56 finished with value: 1473.660400390625 and parameters: {'n_layers': 3, 'n_epochs': 6, 'n_units_l0': 690, 'n_units_l1': 186, 'n_units_l2': 57, 'lr': 0.0010193048967533887, 'n_batchsize': 148}. Best is trial 50 with value: 1282.2197265625.
[I 2022-02-03 17:54:07,659] Trial 57 finished with value: 1419.4273681640625 and parameters: {'n_layers': 3, 'n_epochs': 6, 'n_units_l0': 1354, 'n_units_l1': 238, 'n_units_l2': 81, 'lr': 0.0010969528843021476, 'n_batchsize': 142}. Best is trial 50 with value: 1282.2197265625.
[I 2022-02-03 17:54:09,701] Trial 58 finished with value: 1654.565185546875 and parameters: {'n_layers': 2, 'n_epochs': 4, 'n_units_l0': 1817, 'n_units_l1': 109, 'lr': 0.0002394813794833179, 'n_batchsize': 172}. Best is trial 50 with value: 1282.2197265625.
[I 2022-02-03 17:54:13,318] Trial 59 finished with value: 1783.811767578125 and parameters: {'n_layers': 2, 'n_epochs': 7, 'n_units_l0': 986, 'n_units_l1': 167, 'lr': 0.00774941969651713, 'n_batchsize': 185}. Best is trial 50 with value: 1282.2197265625.
[I 2022-02-03 17:54:16,382] Trial 60 finished with value: 2399.505859375 and parameters: {'n_layers': 1, 'n_epochs': 7, 'n_units_l0': 32, 'lr': 0.0008710613746117566, 'n_batchsize': 159}. Best is trial 50 with value: 1282.2197265625.
[I 2022-02-03 17:54:20,093] Trial 61 finished with value: 1458.66796875 and parameters: {'n_layers': 2, 'n_epochs': 7, 'n_units_l0': 1026, 'n_units_l1': 350, 'lr': 0.0030119323673188153, 'n_batchsize': 144}. Best is trial 50 with value: 1282.2197265625.
[I 2022-02-03 17:54:24,066] Trial 62 finished with value: 1995.706787109375 and parameters: {'n_layers': 2, 'n_epochs': 8, 'n_units_l0': 1312, 'n_units_l1': 11, 'lr': 0.002230704963553553, 'n_batchsize': 167}. Best is trial 50 with value: 1282.2197265625.
[I 2022-02-03 17:54:27,466] Trial 63 finished with value: 1671.4814453125 and parameters: {'n_layers': 2, 'n_epochs': 6, 'n_units_l0': 702, 'n_units_l1': 234, 'lr': 0.01671089593122645, 'n_batchsize': 139}. Best is trial 50 with value: 1282.2197265625.
[I 2022-02-03 17:54:32,131] Trial 64 finished with value: 1611.265625 and parameters: {'n_layers': 2, 'n_epochs': 8, 'n_units_l0': 944, 'n_units_l1': 678, 'lr': 0.006260131989337657, 'n_batchsize': 129}. Best is trial 50 with value: 1282.2197265625.
[I 2022-02-03 17:54:34,952] Trial 65 finished with value: 1273.8626708984375 and parameters: {'n_layers': 2, 'n_epochs': 7, 'n_units_l0': 1976, 'n_units_l1': 348, 'lr': 0.0013214861473761334, 'n_batchsize': 206}. Best is trial 65 with value: 1273.8626708984375.
[I 2022-02-03 17:54:38,050] Trial 66 finished with value: 1480.5250244140625 and parameters: {'n_layers': 3, 'n_epochs': 7, 'n_units_l0': 1741, 'n_units_l1': 174, 'n_units_l2': 28, 'lr': 0.00046235001247300683, 'n_batchsize': 206}. Best is trial 65 with value: 1273.8626708984375.
[I 2022-02-03 17:54:40,888] Trial 67 finished with value: 1473.5164794921875 and parameters: {'n_layers': 2, 'n_epochs': 6, 'n_units_l0': 1384, 'n_units_l1': 315, 'lr': 0.0003339940517305766, 'n_batchsize': 182}. Best is trial 65 with value: 1273.8626708984375.
[I 2022-02-03 17:54:42,844] Trial 68 finished with value: 1633.847900390625 and parameters: {'n_layers': 2, 'n_epochs': 5, 'n_units_l0': 1645, 'n_units_l1': 24, 'lr': 0.0013019617832609178, 'n_batchsize': 242}. Best is trial 65 with value: 1273.8626708984375.
[I 2022-02-03 17:54:45,421] Trial 69 finished with value: 1505.3856201171875 and parameters: {'n_layers': 2, 'n_epochs': 6, 'n_units_l0': 1284, 'n_units_l1': 253, 'lr': 0.004081894362549858, 'n_batchsize': 189}. Best is trial 65 with value: 1273.8626708984375.
[I 2022-02-03 17:54:49,228] Trial 70 finished with value: 1282.8385009765625 and parameters: {'n_layers': 3, 'n_epochs': 7, 'n_units_l0': 1903, 'n_units_l1': 99, 'n_units_l2': 183, 'lr': 0.0008914045999657101, 'n_batchsize': 158}. Best is trial 65 with value: 1273.8626708984375.
[I 2022-02-03 17:54:53,078] Trial 71 finished with value: 1445.5555419921875 and parameters: {'n_layers': 3, 'n_epochs': 7, 'n_units_l0': 1932, 'n_units_l1': 101, 'n_units_l2': 198, 'lr': 0.0008436052475981654, 'n_batchsize': 159}. Best is trial 65 with value: 1273.8626708984375.
[I 2022-02-03 17:54:56,248] Trial 72 finished with value: 1401.774658203125 and parameters: {'n_layers': 2, 'n_epochs': 7, 'n_units_l0': 1188, 'n_units_l1': 53, 'lr': 0.0020715092341148564, 'n_batchsize': 174}. Best is trial 65 with value: 1273.8626708984375.
[I 2022-02-03 17:54:59,776] Trial 73 finished with value: 1340.461181640625 and parameters: {'n_layers': 2, 'n_epochs': 7, 'n_units_l0': 1982, 'n_units_l1': 130, 'lr': 0.0012776827973064245, 'n_batchsize': 154}. Best is trial 65 with value: 1273.8626708984375.
[I 2022-02-03 17:55:02,602] Trial 74 finished with value: 1251.5509033203125 and parameters: {'n_layers': 3, 'n_epochs': 6, 'n_units_l0': 1935, 'n_units_l1': 63, 'n_units_l2': 1965, 'lr': 0.0005604931099750086, 'n_batchsize': 196}. Best is trial 74 with value: 1251.5509033203125.
[I 2022-02-03 17:55:04,892] Trial 75 finished with value: 1266.3035888671875 and parameters: {'n_layers': 3, 'n_epochs': 5, 'n_units_l0': 1120, 'n_units_l1': 67, 'n_units_l2': 965, 'lr': 0.0005796684117723865, 'n_batchsize': 209}. Best is trial 74 with value: 1251.5509033203125.
[I 2022-02-03 17:55:06,950] Trial 76 finished with value: 1418.297119140625 and parameters: {'n_layers': 3, 'n_epochs': 4, 'n_units_l0': 1117, 'n_units_l1': 57, 'n_units_l2': 972, 'lr': 0.0005664276316642517, 'n_batchsize': 202}. Best is trial 74 with value: 1251.5509033203125.
[I 2022-02-03 17:55:09,892] Trial 77 finished with value: 1475.4525146484375 and parameters: {'n_layers': 3, 'n_epochs': 5, 'n_units_l0': 796, 'n_units_l1': 93, 'n_units_l2': 1873, 'lr': 0.00018711590315573898, 'n_batchsize': 210}. Best is trial 74 with value: 1251.5509033203125.
[I 2022-02-03 17:55:11,990] Trial 78 finished with value: 1506.587646484375 and parameters: {'n_layers': 3, 'n_epochs': 4, 'n_units_l0': 1630, 'n_units_l1': 40, 'n_units_l2': 889, 'lr': 0.0002993281420874524, 'n_batchsize': 192}. Best is trial 74 with value: 1251.5509033203125.
[I 2022-02-03 17:55:14,647] Trial 79 finished with value: 2581.775634765625 and parameters: {'n_layers': 4, 'n_epochs': 5, 'n_units_l0': 1132, 'n_units_l1': 64, 'n_units_l2': 435, 'n_units_l3': 11, 'lr': 0.0008048769093424199, 'n_batchsize': 197}. Best is trial 74 with value: 1251.5509033203125.
[I 2022-02-03 17:55:17,173] Trial 80 finished with value: 1276.9898681640625 and parameters: {'n_layers': 3, 'n_epochs': 6, 'n_units_l0': 1714, 'n_units_l1': 46, 'n_units_l2': 1366, 'lr': 0.00039666651959498856, 'n_batchsize': 230}. Best is trial 74 with value: 1251.5509033203125.
[I 2022-02-03 17:55:20,215] Trial 81 finished with value: 1252.2135009765625 and parameters: {'n_layers': 3, 'n_epochs': 6, 'n_units_l0': 1624, 'n_units_l1': 40, 'n_units_l2': 1934, 'lr': 0.00042606550103337873, 'n_batchsize': 178}. Best is trial 74 with value: 1251.5509033203125.
[I 2022-02-03 17:55:22,683] Trial 82 finished with value: 1620.1478271484375 and parameters: {'n_layers': 3, 'n_epochs': 6, 'n_units_l0': 1733, 'n_units_l1': 38, 'n_units_l2': 1354, 'lr': 0.00010613442154781925, 'n_batchsize': 231}. Best is trial 74 with value: 1251.5509033203125.
[I 2022-02-03 17:55:25,911] Trial 83 finished with value: 1221.4814453125 and parameters: {'n_layers': 3, 'n_epochs': 6, 'n_units_l0': 1640, 'n_units_l1': 45, 'n_units_l2': 1825, 'lr': 0.0003799852736147766, 'n_batchsize': 175}. Best is trial 83 with value: 1221.4814453125.
[I 2022-02-03 17:55:28,967] Trial 84 finished with value: 1269.1846923828125 and parameters: {'n_layers': 3, 'n_epochs': 6, 'n_units_l0': 1661, 'n_units_l1': 43, 'n_units_l2': 1993, 'lr': 0.0004056781820212315, 'n_batchsize': 178}. Best is trial 83 with value: 1221.4814453125.
[I 2022-02-03 17:55:31,322] Trial 85 finished with value: 1507.167236328125 and parameters: {'n_layers': 3, 'n_epochs': 6, 'n_units_l0': 1605, 'n_units_l1': 42, 'n_units_l2': 1764, 'lr': 0.00016837161082904138, 'n_batchsize': 253}. Best is trial 83 with value: 1221.4814453125.
[I 2022-02-03 17:55:34,040] Trial 86 finished with value: 1379.84375 and parameters: {'n_layers': 3, 'n_epochs': 5, 'n_units_l0': 1855, 'n_units_l1': 29, 'n_units_l2': 1928, 'lr': 0.00026581908446809873, 'n_batchsize': 180}. Best is trial 83 with value: 1221.4814453125.
[I 2022-02-03 17:55:36,810] Trial 87 finished with value: 1233.6475830078125 and parameters: {'n_layers': 3, 'n_epochs': 6, 'n_units_l0': 1975, 'n_units_l1': 71, 'n_units_l2': 1172, 'lr': 0.00038925702085134957, 'n_batchsize': 212}. Best is trial 83 with value: 1221.4814453125.
[I 2022-02-03 17:55:39,089] Trial 88 finished with value: 1363.4786376953125 and parameters: {'n_layers': 3, 'n_epochs': 5, 'n_units_l0': 1540, 'n_units_l1': 33, 'n_units_l2': 1108, 'lr': 0.0004072841550363365, 'n_batchsize': 211}. Best is trial 83 with value: 1221.4814453125.
[I 2022-02-03 17:55:42,173] Trial 89 finished with value: 1383.331787109375 and parameters: {'n_layers': 4, 'n_epochs': 6, 'n_units_l0': 1530, 'n_units_l1': 22, 'n_units_l2': 1311, 'n_units_l3': 32, 'lr': 0.0005789691157084724, 'n_batchsize': 194}. Best is trial 83 with value: 1221.4814453125.
[I 2022-02-03 17:55:44,725] Trial 90 finished with value: 1550.879638671875 and parameters: {'n_layers': 3, 'n_epochs': 6, 'n_units_l0': 1253, 'n_units_l1': 72, 'n_units_l2': 1448, 'lr': 0.000138604441237174, 'n_batchsize': 237}. Best is trial 83 with value: 1221.4814453125.
[I 2022-02-03 17:55:47,898] Trial 91 finished with value: 1272.9847412109375 and parameters: {'n_layers': 3, 'n_epochs': 6, 'n_units_l0': 1951, 'n_units_l1': 46, 'n_units_l2': 1003, 'lr': 0.0003860785536124425, 'n_batchsize': 173}. Best is trial 83 with value: 1221.4814453125.
[I 2022-02-03 17:55:50,871] Trial 92 finished with value: 1257.542236328125 and parameters: {'n_layers': 3, 'n_epochs': 6, 'n_units_l0': 1686, 'n_units_l1': 46, 'n_units_l2': 1071, 'lr': 0.00038931471048207056, 'n_batchsize': 175}. Best is trial 83 with value: 1221.4814453125.
[I 2022-02-03 17:55:53,634] Trial 93 finished with value: 1295.67626953125 and parameters: {'n_layers': 3, 'n_epochs': 5, 'n_units_l0': 1726, 'n_units_l1': 47, 'n_units_l2': 1008, 'lr': 0.00038299278770520767, 'n_batchsize': 171}. Best is trial 83 with value: 1221.4814453125.
[I 2022-02-03 17:55:57,242] Trial 94 finished with value: 1278.9736328125 and parameters: {'n_layers': 3, 'n_epochs': 6, 'n_units_l0': 1968, 'n_units_l1': 62, 'n_units_l2': 1983, 'lr': 0.0002891977296844797, 'n_batchsize': 183}. Best is trial 83 with value: 1221.4814453125.
[I 2022-02-03 17:56:00,000] Trial 95 finished with value: 1453.6832275390625 and parameters: {'n_layers': 3, 'n_epochs': 6, 'n_units_l0': 1998, 'n_units_l1': 32, 'n_units_l2': 1523, 'lr': 0.00020595881792311, 'n_batchsize': 216}. Best is trial 83 with value: 1221.4814453125.
[I 2022-02-03 17:56:02,168] Trial 96 finished with value: 1314.328125 and parameters: {'n_layers': 3, 'n_epochs': 5, 'n_units_l0': 1481, 'n_units_l1': 46, 'n_units_l2': 1242, 'lr': 0.0004914300991837485, 'n_batchsize': 231}. Best is trial 83 with value: 1221.4814453125.
[I 2022-02-03 17:56:04,905] Trial 97 finished with value: 1624.525146484375 and parameters: {'n_layers': 3, 'n_epochs': 6, 'n_units_l0': 1178, 'n_units_l1': 72, 'n_units_l2': 874, 'lr': 9.733763813416803e-05, 'n_batchsize': 201}. Best is trial 83 with value: 1221.4814453125.
[I 2022-02-03 17:56:07,247] Trial 98 finished with value: 1261.6739501953125 and parameters: {'n_layers': 3, 'n_epochs': 5, 'n_units_l0': 1714, 'n_units_l1': 82, 'n_units_l2': 1557, 'lr': 0.0006536280920710938, 'n_batchsize': 210}. Best is trial 83 with value: 1221.4814453125.
[I 2022-02-03 17:56:09,051] Trial 99 finished with value: 1309.45947265625 and parameters: {'n_layers': 3, 'n_epochs': 4, 'n_units_l0': 1507, 'n_units_l1': 81, 'n_units_l2': 1629, 'lr': 0.0006484508891321097, 'n_batchsize': 247}. Best is trial 83 with value: 1221.4814453125.
[I 2022-02-03 17:56:11,747] Trial 100 finished with value: 1261.9166259765625 and parameters: {'n_layers': 3, 'n_epochs': 5, 'n_units_l0': 1089, 'n_units_l1': 57, 'n_units_l2': 1089, 'lr': 0.0004957444426539617, 'n_batchsize': 179}. Best is trial 83 with value: 1221.4814453125.
[I 2022-02-03 17:56:14,416] Trial 101 finished with value: 1605.4219970703125 and parameters: {'n_layers': 3, 'n_epochs': 5, 'n_units_l0': 62, 'n_units_l1': 57, 'n_units_l2': 1234, 'lr': 0.00032996657197630796, 'n_batchsize': 178}. Best is trial 83 with value: 1221.4814453125.
[I 2022-02-03 17:56:16,839] Trial 102 finished with value: 1332.9237060546875 and parameters: {'n_layers': 3, 'n_epochs': 5, 'n_units_l0': 1085, 'n_units_l1': 34, 'n_units_l2': 840, 'lr': 0.0004657529039721339, 'n_batchsize': 188}. Best is trial 83 with value: 1221.4814453125.
[I 2022-02-03 17:56:19,583] Trial 103 finished with value: 1379.1165771484375 and parameters: {'n_layers': 3, 'n_epochs': 5, 'n_units_l0': 1741, 'n_units_l1': 87, 'n_units_l2': 1080, 'lr': 0.00023005755282278013, 'n_batchsize': 166}. Best is trial 83 with value: 1221.4814453125.
[I 2022-02-03 17:56:22,256] Trial 104 finished with value: 1275.086669921875 and parameters: {'n_layers': 3, 'n_epochs': 6, 'n_units_l0': 1315, 'n_units_l1': 66, 'n_units_l2': 1592, 'lr': 0.000677433749760155, 'n_batchsize': 208}. Best is trial 83 with value: 1221.4814453125.
[I 2022-02-03 17:56:24,512] Trial 105 finished with value: 1616.36767578125 and parameters: {'n_layers': 4, 'n_epochs': 4, 'n_units_l0': 33, 'n_units_l1': 51, 'n_units_l2': 601, 'n_units_l3': 53, 'lr': 0.0005578753873139754, 'n_batchsize': 195}. Best is trial 83 with value: 1221.4814453125.
[I 2022-02-03 17:56:26,762] Trial 106 finished with value: 1373.0128173828125 and parameters: {'n_layers': 3, 'n_epochs': 5, 'n_units_l0': 1247, 'n_units_l1': 36, 'n_units_l2': 1957, 'lr': 0.0003607451461530265, 'n_batchsize': 215}. Best is trial 83 with value: 1221.4814453125.
[I 2022-02-03 17:56:28,571] Trial 107 finished with value: 1891.94384765625 and parameters: {'n_layers': 3, 'n_epochs': 6, 'n_units_l0': 218, 'n_units_l1': 24, 'n_units_l2': 1152, 'lr': 0.0001696267849411591, 'n_batchsize': 384}. Best is trial 83 with value: 1221.4814453125.
[I 2022-02-03 17:56:31,151] Trial 108 finished with value: 1438.20556640625 and parameters: {'n_layers': 3, 'n_epochs': 5, 'n_units_l0': 1504, 'n_units_l1': 43, 'n_units_l2': 724, 'lr': 0.0002622388647871342, 'n_batchsize': 178}. Best is trial 83 with value: 1221.4814453125.
[I 2022-02-03 17:56:34,368] Trial 109 finished with value: 1641.966552734375 and parameters: {'n_layers': 3, 'n_epochs': 6, 'n_units_l0': 12, 'n_units_l1': 57, 'n_units_l2': 1589, 'lr': 0.0004878706214079707, 'n_batchsize': 165}. Best is trial 83 with value: 1221.4814453125.
[I 2022-02-03 17:56:37,662] Trial 110 finished with value: 1596.2137451171875 and parameters: {'n_layers': 3, 'n_epochs': 6, 'n_units_l0': 1059, 'n_units_l1': 16, 'n_units_l2': 551, 'lr': 0.0007407672509666422, 'n_batchsize': 202}. Best is trial 83 with value: 1221.4814453125.
[I 2022-02-03 17:56:40,395] Trial 111 finished with value: 1246.421142578125 and parameters: {'n_layers': 3, 'n_epochs': 6, 'n_units_l0': 1762, 'n_units_l1': 66, 'n_units_l2': 1600, 'lr': 0.0006720143632739736, 'n_batchsize': 211}. Best is trial 83 with value: 1221.4814453125.
[I 2022-02-03 17:56:43,257] Trial 112 finished with value: 1261.6439208984375 and parameters: {'n_layers': 3, 'n_epochs': 6, 'n_units_l0': 1797, 'n_units_l1': 113, 'n_units_l2': 1591, 'lr': 0.00044645934273222245, 'n_batchsize': 191}. Best is trial 83 with value: 1221.4814453125.
[I 2022-02-03 17:56:46,319] Trial 113 finished with value: 1269.5640869140625 and parameters: {'n_layers': 3, 'n_epochs': 6, 'n_units_l0': 1756, 'n_units_l1': 114, 'n_units_l2': 1601, 'lr': 0.00031597163282929576, 'n_batchsize': 186}. Best is trial 83 with value: 1221.4814453125.
[I 2022-02-03 17:56:49,367] Trial 114 finished with value: 1261.38720703125 and parameters: {'n_layers': 3, 'n_epochs': 6, 'n_units_l0': 1685, 'n_units_l1': 115, 'n_units_l2': 1621, 'lr': 0.0003138603249632897, 'n_batchsize': 188}. Best is trial 83 with value: 1221.4814453125.
[I 2022-02-03 17:56:51,613] Trial 115 finished with value: 1298.01806640625 and parameters: {'n_layers': 3, 'n_epochs': 5, 'n_units_l0': 1450, 'n_units_l1': 154, 'n_units_l2': 1985, 'lr': 0.0006008157462513592, 'n_batchsize': 220}. Best is trial 83 with value: 1221.4814453125.
[I 2022-02-03 17:56:54,602] Trial 116 finished with value: 1215.2210693359375 and parameters: {'n_layers': 3, 'n_epochs': 6, 'n_units_l0': 1635, 'n_units_l1': 86, 'n_units_l2': 1414, 'lr': 0.00044206624585741274, 'n_batchsize': 189}. Best is trial 116 with value: 1215.2210693359375.
[I 2022-02-03 17:56:57,567] Trial 117 finished with value: 1324.336181640625 and parameters: {'n_layers': 3, 'n_epochs': 6, 'n_units_l0': 893, 'n_units_l1': 86, 'n_units_l2': 1407, 'lr': 0.00046421151844742824, 'n_batchsize': 200}. Best is trial 116 with value: 1215.2210693359375.
[I 2022-02-03 17:57:00,093] Trial 118 finished with value: 1869.920654296875 and parameters: {'n_layers': 3, 'n_epochs': 5, 'n_units_l0': 1235, 'n_units_l1': 77, 'n_units_l2': 1211, 'lr': 4.641118674904737e-05, 'n_batchsize': 191}. Best is trial 116 with value: 1215.2210693359375.
[I 2022-02-03 17:57:02,490] Trial 119 finished with value: 1412.46337890625 and parameters: {'n_layers': 3, 'n_epochs': 6, 'n_units_l0': 1408, 'n_units_l1': 125, 'n_units_l2': 1600, 'lr': 0.00020931211325866386, 'n_batchsize': 264}. Best is trial 116 with value: 1215.2210693359375.
[I 2022-02-03 17:57:04,700] Trial 120 finished with value: 1282.8616943359375 and parameters: {'n_layers': 3, 'n_epochs': 5, 'n_units_l0': 1609, 'n_units_l1': 68, 'n_units_l2': 1334, 'lr': 0.0007719917329855254, 'n_batchsize': 225}. Best is trial 116 with value: 1215.2210693359375.
[I 2022-02-03 17:57:07,620] Trial 121 finished with value: 1225.8076171875 and parameters: {'n_layers': 3, 'n_epochs': 6, 'n_units_l0': 1747, 'n_units_l1': 53, 'n_units_l2': 1703, 'lr': 0.0005247778387467126, 'n_batchsize': 186}. Best is trial 116 with value: 1215.2210693359375.
[I 2022-02-03 17:57:10,563] Trial 122 finished with value: 1252.0013427734375 and parameters: {'n_layers': 3, 'n_epochs': 6, 'n_units_l0': 1769, 'n_units_l1': 94, 'n_units_l2': 1688, 'lr': 0.0005230345781373263, 'n_batchsize': 187}. Best is trial 116 with value: 1215.2210693359375.
[I 2022-02-03 17:57:13,469] Trial 123 finished with value: 1292.668212890625 and parameters: {'n_layers': 3, 'n_epochs': 6, 'n_units_l0': 1762, 'n_units_l1': 111, 'n_units_l2': 1683, 'lr': 0.0002873294439776751, 'n_batchsize': 185}. Best is trial 116 with value: 1215.2210693359375.
[I 2022-02-03 17:57:16,295] Trial 124 finished with value: 1217.9534912109375 and parameters: {'n_layers': 3, 'n_epochs': 6, 'n_units_l0': 1587, 'n_units_l1': 87, 'n_units_l2': 1424, 'lr': 0.0005055312207340312, 'n_batchsize': 191}. Best is trial 116 with value: 1215.2210693359375.
[I 2022-02-03 17:57:19,168] Trial 125 finished with value: 152167.96875 and parameters: {'n_layers': 3, 'n_epochs': 6, 'n_units_l0': 1768, 'n_units_l1': 100, 'n_units_l2': 1669, 'lr': 0.05367486439028279, 'n_batchsize': 195}. Best is trial 116 with value: 1215.2210693359375.
[I 2022-02-03 17:57:22,534] Trial 126 finished with value: 1269.4732666015625 and parameters: {'n_layers': 3, 'n_epochs': 6, 'n_units_l0': 1413, 'n_units_l1': 143, 'n_units_l2': 1453, 'lr': 0.0006461013733485506, 'n_batchsize': 163}. Best is trial 116 with value: 1215.2210693359375.
[I 2022-02-03 17:57:25,148] Trial 127 finished with value: 1328.4315185546875 and parameters: {'n_layers': 3, 'n_epochs': 6, 'n_units_l0': 1591, 'n_units_l1': 85, 'n_units_l2': 1803, 'lr': 0.0003567408392117129, 'n_batchsize': 338}. Best is trial 116 with value: 1215.2210693359375.
[I 2022-02-03 17:57:28,232] Trial 128 finished with value: 1277.5882568359375 and parameters: {'n_layers': 3, 'n_epochs': 6, 'n_units_l0': 1280, 'n_units_l1': 94, 'n_units_l2': 1389, 'lr': 0.0009497912521813967, 'n_batchsize': 189}. Best is trial 116 with value: 1215.2210693359375.
[I 2022-02-03 17:57:31,006] Trial 129 finished with value: 1279.7054443359375 and parameters: {'n_layers': 3, 'n_epochs': 6, 'n_units_l0': 1816, 'n_units_l1': 77, 'n_units_l2': 1741, 'lr': 0.0002654187180614154, 'n_batchsize': 213}. Best is trial 116 with value: 1215.2210693359375.
[I 2022-02-03 17:57:33,044] Trial 130 finished with value: 1282.887939453125 and parameters: {'n_layers': 3, 'n_epochs': 6, 'n_units_l0': 1569, 'n_units_l1': 118, 'n_units_l2': 1217, 'lr': 0.0004353580039070643, 'n_batchsize': 306}. Best is trial 116 with value: 1215.2210693359375.
[I 2022-02-03 17:57:36,269] Trial 131 finished with value: 1247.33154296875 and parameters: {'n_layers': 3, 'n_epochs': 6, 'n_units_l0': 1972, 'n_units_l1': 59, 'n_units_l2': 1100, 'lr': 0.0005238963216363616, 'n_batchsize': 170}. Best is trial 116 with value: 1215.2210693359375.
[I 2022-02-03 17:57:39,465] Trial 132 finished with value: 1259.90185546875 and parameters: {'n_layers': 3, 'n_epochs': 6, 'n_units_l0': 1971, 'n_units_l1': 62, 'n_units_l2': 1506, 'lr': 0.000717304607322428, 'n_batchsize': 170}. Best is trial 116 with value: 1215.2210693359375.
[I 2022-02-03 17:57:43,122] Trial 133 finished with value: 1212.3564453125 and parameters: {'n_layers': 3, 'n_epochs': 7, 'n_units_l0': 1936, 'n_units_l1': 52, 'n_units_l2': 1365, 'lr': 0.0007888718344970752, 'n_batchsize': 169}. Best is trial 133 with value: 1212.3564453125.
[I 2022-02-03 17:57:46,706] Trial 134 finished with value: 1270.123046875 and parameters: {'n_layers': 3, 'n_epochs': 7, 'n_units_l0': 1982, 'n_units_l1': 61, 'n_units_l2': 1358, 'lr': 0.0005467890740781994, 'n_batchsize': 171}. Best is trial 133 with value: 1212.3564453125.
[I 2022-02-03 17:57:50,813] Trial 135 finished with value: 1295.09326171875 and parameters: {'n_layers': 3, 'n_epochs': 7, 'n_units_l0': 1448, 'n_units_l1': 51, 'n_units_l2': 799, 'lr': 0.0008195184552887825, 'n_batchsize': 141}. Best is trial 133 with value: 1212.3564453125.
[I 2022-02-03 17:57:54,584] Trial 136 finished with value: 1255.8001708984375 and parameters: {'n_layers': 3, 'n_epochs': 7, 'n_units_l0': 1527, 'n_units_l1': 73, 'n_units_l2': 999, 'lr': 0.00033470984535912166, 'n_batchsize': 157}. Best is trial 133 with value: 1212.3564453125.
[I 2022-02-03 17:57:58,251] Trial 137 finished with value: 1243.9130859375 and parameters: {'n_layers': 3, 'n_epochs': 7, 'n_units_l0': 1997, 'n_units_l1': 72, 'n_units_l2': 973, 'lr': 0.0007397145748599099, 'n_batchsize': 161}. Best is trial 133 with value: 1212.3564453125.
[I 2022-02-03 17:58:02,007] Trial 138 finished with value: 1398.367431640625 and parameters: {'n_layers': 3, 'n_epochs': 7, 'n_units_l0': 1317, 'n_units_l1': 72, 'n_units_l2': 959, 'lr': 0.0009896823263576041, 'n_batchsize': 159}. Best is trial 133 with value: 1212.3564453125.
[I 2022-02-03 17:58:06,189] Trial 139 finished with value: 1201.6868896484375 and parameters: {'n_layers': 3, 'n_epochs': 7, 'n_units_l0': 1579, 'n_units_l1': 53, 'n_units_l2': 1112, 'lr': 0.0003571932851669267, 'n_batchsize': 140}. Best is trial 139 with value: 1201.6868896484375.
[I 2022-02-03 17:58:10,065] Trial 140 finished with value: 1243.52294921875 and parameters: {'n_layers': 3, 'n_epochs': 7, 'n_units_l0': 1516, 'n_units_l1': 53, 'n_units_l2': 1173, 'lr': 0.0005357823859238149, 'n_batchsize': 155}. Best is trial 139 with value: 1201.6868896484375.
[I 2022-02-03 17:58:14,154] Trial 141 finished with value: 1256.9647216796875 and parameters: {'n_layers': 3, 'n_epochs': 7, 'n_units_l0': 1534, 'n_units_l1': 54, 'n_units_l2': 1180, 'lr': 0.0005458003009714545, 'n_batchsize': 138}. Best is trial 139 with value: 1201.6868896484375.
[I 2022-02-03 17:58:18,021] Trial 142 finished with value: 1219.7066650390625 and parameters: {'n_layers': 3, 'n_epochs': 7, 'n_units_l0': 1847, 'n_units_l1': 39, 'n_units_l2': 990, 'lr': 0.00035693053431035024, 'n_batchsize': 154}. Best is trial 139 with value: 1201.6868896484375.
[I 2022-02-03 17:58:22,432] Trial 143 finished with value: 1537.09521484375 and parameters: {'n_layers': 3, 'n_epochs': 8, 'n_units_l0': 1837, 'n_units_l1': 40, 'n_units_l2': 13, 'lr': 0.0005387400746246176, 'n_batchsize': 149}. Best is trial 139 with value: 1201.6868896484375.
[I 2022-02-03 17:58:27,370] Trial 144 finished with value: 1487.491455078125 and parameters: {'n_layers': 3, 'n_epochs': 7, 'n_units_l0': 107, 'n_units_l1': 52, 'n_units_l2': 1309, 'lr': 0.0008157667829941372, 'n_batchsize': 136}. Best is trial 139 with value: 1201.6868896484375.
[I 2022-02-03 17:58:31,114] Trial 145 finished with value: 1339.0802001953125 and parameters: {'n_layers': 3, 'n_epochs': 7, 'n_units_l0': 1991, 'n_units_l1': 37, 'n_units_l2': 715, 'lr': 0.00023630598249034746, 'n_batchsize': 163}. Best is trial 139 with value: 1201.6868896484375.
[I 2022-02-03 17:58:35,276] Trial 146 finished with value: 1251.908935546875 and parameters: {'n_layers': 3, 'n_epochs': 7, 'n_units_l0': 1335, 'n_units_l1': 61, 'n_units_l2': 871, 'lr': 0.0004344716276414582, 'n_batchsize': 143}. Best is trial 139 with value: 1201.6868896484375.
[I 2022-02-03 17:58:39,239] Trial 147 finished with value: 1345.583740234375 and parameters: {'n_layers': 3, 'n_epochs': 7, 'n_units_l0': 1382, 'n_units_l1': 63, 'n_units_l2': 894, 'lr': 0.001229323471996121, 'n_batchsize': 146}. Best is trial 139 with value: 1201.6868896484375.
[I 2022-02-03 17:58:43,250] Trial 148 finished with value: 1319.2835693359375 and parameters: {'n_layers': 3, 'n_epochs': 7, 'n_units_l0': 1195, 'n_units_l1': 50, 'n_units_l2': 1132, 'lr': 0.0006757598480458637, 'n_batchsize': 155}. Best is trial 139 with value: 1201.6868896484375.
[I 2022-02-03 17:58:48,090] Trial 149 finished with value: 2008.0191650390625 and parameters: {'n_layers': 3, 'n_epochs': 8, 'n_units_l0': 1401, 'n_units_l1': 57, 'n_units_l2': 1023, 'lr': 1.9038014544171616e-05, 'n_batchsize': 137}. Best is trial 139 with value: 1201.6868896484375.
[I 2022-02-03 17:58:52,204] Trial 150 finished with value: 1254.03076171875 and parameters: {'n_layers': 3, 'n_epochs': 7, 'n_units_l0': 1636, 'n_units_l1': 68, 'n_units_l2': 256, 'lr': 0.0005133601194372962, 'n_batchsize': 144}. Best is trial 139 with value: 1201.6868896484375.
[I 2022-02-03 17:58:55,906] Trial 151 finished with value: 1242.1357421875 and parameters: {'n_layers': 3, 'n_epochs': 7, 'n_units_l0': 1609, 'n_units_l1': 48, 'n_units_l2': 1822, 'lr': 0.00041790520815698274, 'n_batchsize': 168}. Best is trial 139 with value: 1201.6868896484375.
[I 2022-02-03 17:58:59,656] Trial 152 finished with value: 1227.70361328125 and parameters: {'n_layers': 3, 'n_epochs': 7, 'n_units_l0': 1821, 'n_units_l1': 48, 'n_units_l2': 799, 'lr': 0.0003756796372663255, 'n_batchsize': 166}. Best is trial 139 with value: 1201.6868896484375.
[I 2022-02-03 17:59:02,996] Trial 153 finished with value: 1219.0582275390625 and parameters: {'n_layers': 3, 'n_epochs': 7, 'n_units_l0': 1847, 'n_units_l1': 46, 'n_units_l2': 888, 'lr': 0.0003586945477851318, 'n_batchsize': 160}. Best is trial 139 with value: 1201.6868896484375.
[I 2022-02-03 17:59:06,657] Trial 154 finished with value: 1245.2646484375 and parameters: {'n_layers': 3, 'n_epochs': 7, 'n_units_l0': 1968, 'n_units_l1': 47, 'n_units_l2': 816, 'lr': 0.0003281945343811341, 'n_batchsize': 165}. Best is trial 139 with value: 1201.6868896484375.
[I 2022-02-03 17:59:10,401] Trial 155 finished with value: 1221.20166015625 and parameters: {'n_layers': 3, 'n_epochs': 7, 'n_units_l0': 1979, 'n_units_l1': 44, 'n_units_l2': 909, 'lr': 0.0003449112207752404, 'n_batchsize': 166}. Best is trial 139 with value: 1201.6868896484375.
[I 2022-02-03 17:59:14,825] Trial 156 finished with value: 1241.93408203125 and parameters: {'n_layers': 3, 'n_epochs': 8, 'n_units_l0': 1572, 'n_units_l1': 32, 'n_units_l2': 786, 'lr': 0.0002742953971413993, 'n_batchsize': 154}. Best is trial 139 with value: 1201.6868896484375.
[I 2022-02-03 17:59:18,973] Trial 157 finished with value: 1420.731201171875 and parameters: {'n_layers': 3, 'n_epochs': 8, 'n_units_l0': 1567, 'n_units_l1': 31, 'n_units_l2': 808, 'lr': 0.00016307885274394491, 'n_batchsize': 155}. Best is trial 139 with value: 1201.6868896484375.
[I 2022-02-03 17:59:23,234] Trial 158 finished with value: 1231.427734375 and parameters: {'n_layers': 5, 'n_epochs': 8, 'n_units_l0': 1990, 'n_units_l1': 28, 'n_units_l2': 686, 'n_units_l3': 1830, 'n_units_l4': 1729, 'lr': 0.00020885092328852122, 'n_batchsize': 160}. Best is trial 139 with value: 1201.6868896484375.
[I 2022-02-03 17:59:29,089] Trial 159 finished with value: 1260.3590087890625 and parameters: {'n_layers': 5, 'n_epochs': 8, 'n_units_l0': 1497, 'n_units_l1': 26, 'n_units_l2': 665, 'n_units_l3': 1764, 'n_units_l4': 1908, 'lr': 0.00021396447620197436, 'n_batchsize': 128}. Best is trial 139 with value: 1201.6868896484375.
[I 2022-02-03 17:59:34,343] Trial 160 finished with value: 1330.83056640625 and parameters: {'n_layers': 5, 'n_epochs': 8, 'n_units_l0': 1214, 'n_units_l1': 36, 'n_units_l2': 594, 'n_units_l3': 813, 'n_units_l4': 539, 'lr': 0.00012435087144139034, 'n_batchsize': 162}. Best is trial 139 with value: 1201.6868896484375.
[I 2022-02-03 17:59:38,734] Trial 161 finished with value: 1291.986328125 and parameters: {'n_layers': 3, 'n_epochs': 8, 'n_units_l0': 1991, 'n_units_l1': 30, 'n_units_l2': 465, 'lr': 0.0002544795753447116, 'n_batchsize': 150}. Best is trial 139 with value: 1201.6868896484375.
[I 2022-02-03 17:59:42,248] Trial 162 finished with value: 1219.1507568359375 and parameters: {'n_layers': 3, 'n_epochs': 7, 'n_units_l0': 1733, 'n_units_l1': 46, 'n_units_l2': 921, 'lr': 0.0003328798700105666, 'n_batchsize': 167}. Best is trial 139 with value: 1201.6868896484375.
[I 2022-02-03 17:59:46,023] Trial 163 finished with value: 1251.9613037109375 and parameters: {'n_layers': 3, 'n_epochs': 7, 'n_units_l0': 1657, 'n_units_l1': 42, 'n_units_l2': 924, 'lr': 0.00029065171412178016, 'n_batchsize': 156}. Best is trial 139 with value: 1201.6868896484375.
[I 2022-02-03 17:59:49,791] Trial 164 finished with value: 1236.521240234375 and parameters: {'n_layers': 4, 'n_epochs': 7, 'n_units_l0': 1782, 'n_units_l1': 38, 'n_units_l2': 766, 'n_units_l3': 1135, 'lr': 0.00035755310632561756, 'n_batchsize': 170}. Best is trial 139 with value: 1201.6868896484375.
[I 2022-02-03 17:59:53,871] Trial 165 finished with value: 1274.539306640625 and parameters: {'n_layers': 5, 'n_epochs': 7, 'n_units_l0': 1773, 'n_units_l1': 19, 'n_units_l2': 771, 'n_units_l3': 1319, 'n_units_l4': 357, 'lr': 0.0001993932094075294, 'n_batchsize': 172}. Best is trial 139 with value: 1201.6868896484375.
[I 2022-02-03 17:59:58,193] Trial 166 finished with value: 1227.817626953125 and parameters: {'n_layers': 4, 'n_epochs': 8, 'n_units_l0': 1467, 'n_units_l1': 27, 'n_units_l2': 697, 'n_units_l3': 786, 'lr': 0.0003501409497908598, 'n_batchsize': 177}. Best is trial 139 with value: 1201.6868896484375.
[I 2022-02-03 18:00:02,305] Trial 167 finished with value: 1232.684814453125 and parameters: {'n_layers': 4, 'n_epochs': 8, 'n_units_l0': 1383, 'n_units_l1': 26, 'n_units_l2': 543, 'n_units_l3': 630, 'lr': 0.0003595555235425204, 'n_batchsize': 178}. Best is trial 139 with value: 1201.6868896484375.
[I 2022-02-03 18:00:06,393] Trial 168 finished with value: 1238.760498046875 and parameters: {'n_layers': 4, 'n_epochs': 8, 'n_units_l0': 1359, 'n_units_l1': 27, 'n_units_l2': 529, 'n_units_l3': 558, 'lr': 0.00035359119114598185, 'n_batchsize': 179}. Best is trial 139 with value: 1201.6868896484375.
[I 2022-02-03 18:00:10,641] Trial 169 finished with value: 1226.6446533203125 and parameters: {'n_layers': 4, 'n_epochs': 8, 'n_units_l0': 1314, 'n_units_l1': 29, 'n_units_l2': 651, 'n_units_l3': 627, 'lr': 0.0003705131777692496, 'n_batchsize': 182}. Best is trial 139 with value: 1201.6868896484375.
[I 2022-02-03 18:00:13,044] Trial 170 finished with value: 1375.734375 and parameters: {'n_layers': 4, 'n_epochs': 8, 'n_units_l0': 1208, 'n_units_l1': 21, 'n_units_l2': 731, 'n_units_l3': 1188, 'lr': 0.00023335980534401713, 'n_batchsize': 402}. Best is trial 139 with value: 1201.6868896484375.
[I 2022-02-03 18:00:17,329] Trial 171 finished with value: 1219.496337890625 and parameters: {'n_layers': 4, 'n_epochs': 8, 'n_units_l0': 1393, 'n_units_l1': 28, 'n_units_l2': 557, 'n_units_l3': 590, 'lr': 0.00035746775020744747, 'n_batchsize': 175}. Best is trial 139 with value: 1201.6868896484375.
[I 2022-02-03 18:00:21,377] Trial 172 finished with value: 1229.4317626953125 and parameters: {'n_layers': 4, 'n_epochs': 8, 'n_units_l0': 1784, 'n_units_l1': 29, 'n_units_l2': 661, 'n_units_l3': 723, 'lr': 0.00036344618841702695, 'n_batchsize': 180}. Best is trial 139 with value: 1201.6868896484375.
[I 2022-02-03 18:00:25,510] Trial 173 finished with value: 1289.6434326171875 and parameters: {'n_layers': 4, 'n_epochs': 8, 'n_units_l0': 1438, 'n_units_l1': 24, 'n_units_l2': 625, 'n_units_l3': 636, 'lr': 0.00030122859841848497, 'n_batchsize': 182}. Best is trial 139 with value: 1201.6868896484375.
[I 2022-02-03 18:00:29,755] Trial 174 finished with value: 1269.8272705078125 and parameters: {'n_layers': 4, 'n_epochs': 8, 'n_units_l0': 1776, 'n_units_l1': 28, 'n_units_l2': 420, 'n_units_l3': 488, 'lr': 0.00040043237162954684, 'n_batchsize': 177}. Best is trial 139 with value: 1201.6868896484375.
[I 2022-02-03 18:00:34,180] Trial 175 finished with value: 1242.3082275390625 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 997, 'n_units_l1': 16, 'n_units_l2': 587, 'n_units_l3': 832, 'lr': 0.0003563488800363841, 'n_batchsize': 182}. Best is trial 139 with value: 1201.6868896484375.
[I 2022-02-03 18:00:39,042] Trial 176 finished with value: 1274.4693603515625 and parameters: {'n_layers': 4, 'n_epochs': 8, 'n_units_l0': 1299, 'n_units_l1': 21, 'n_units_l2': 370, 'n_units_l3': 1394, 'lr': 0.0001859245558270593, 'n_batchsize': 165}. Best is trial 139 with value: 1201.6868896484375.
[I 2022-02-03 18:00:43,839] Trial 177 finished with value: 1209.6461181640625 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 1758, 'n_units_l1': 34, 'n_units_l2': 469, 'n_units_l3': 780, 'lr': 0.00028872878051995393, 'n_batchsize': 176}. Best is trial 139 with value: 1201.6868896484375.
[I 2022-02-03 18:00:48,640] Trial 178 finished with value: 1243.8668212890625 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 1670, 'n_units_l1': 26, 'n_units_l2': 667, 'n_units_l3': 711, 'lr': 0.0002640240609610731, 'n_batchsize': 175}. Best is trial 139 with value: 1201.6868896484375.
[I 2022-02-03 18:00:53,470] Trial 179 finished with value: 1330.962646484375 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 1436, 'n_units_l1': 35, 'n_units_l2': 520, 'n_units_l3': 199, 'lr': 0.0001588099277104474, 'n_batchsize': 174}. Best is trial 139 with value: 1201.6868896484375.
[I 2022-02-03 18:00:57,449] Trial 180 finished with value: 1261.888671875 and parameters: {'n_layers': 4, 'n_epochs': 8, 'n_units_l0': 1105, 'n_units_l1': 34, 'n_units_l2': 472, 'n_units_l3': 452, 'lr': 0.0003077432221008278, 'n_batchsize': 185}. Best is trial 139 with value: 1201.6868896484375.
[I 2022-02-03 18:01:01,625] Trial 181 finished with value: 1826.50146484375 and parameters: {'n_layers': 5, 'n_epochs': 8, 'n_units_l0': 1791, 'n_units_l1': 29, 'n_units_l2': 709, 'n_units_l3': 909, 'n_units_l4': 10, 'lr': 0.0004039274161843417, 'n_batchsize': 196}. Best is trial 139 with value: 1201.6868896484375.
[I 2022-02-03 18:01:06,146] Trial 182 finished with value: 1242.000732421875 and parameters: {'n_layers': 4, 'n_epochs': 8, 'n_units_l0': 1604, 'n_units_l1': 43, 'n_units_l2': 572, 'n_units_l3': 599, 'lr': 0.00021616162193132654, 'n_batchsize': 164}. Best is trial 139 with value: 1201.6868896484375.
[I 2022-02-03 18:01:10,169] Trial 183 finished with value: 1290.9066162109375 and parameters: {'n_layers': 4, 'n_epochs': 8, 'n_units_l0': 1814, 'n_units_l1': 31, 'n_units_l2': 641, 'n_units_l3': 1575, 'lr': 0.0003208943100939965, 'n_batchsize': 181}. Best is trial 139 with value: 1201.6868896484375.
[I 2022-02-03 18:01:14,874] Trial 184 finished with value: 1209.506591796875 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 1443, 'n_units_l1': 24, 'n_units_l2': 99, 'n_units_l3': 938, 'lr': 0.00045796187114691296, 'n_batchsize': 170}. Best is trial 139 with value: 1201.6868896484375.
[I 2022-02-03 18:01:19,769] Trial 185 finished with value: 1202.6712646484375 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 1414, 'n_units_l1': 18, 'n_units_l2': 134, 'n_units_l3': 961, 'lr': 0.0004562004617178676, 'n_batchsize': 169}. Best is trial 139 with value: 1201.6868896484375.
[I 2022-02-03 18:01:24,634] Trial 186 finished with value: 1213.104736328125 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 1554, 'n_units_l1': 17, 'n_units_l2': 162, 'n_units_l3': 972, 'lr': 0.00045409175109619387, 'n_batchsize': 168}. Best is trial 139 with value: 1201.6868896484375.
[I 2022-02-03 18:01:29,509] Trial 187 finished with value: 1254.3779296875 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 1260, 'n_units_l1': 23, 'n_units_l2': 127, 'n_units_l3': 927, 'lr': 0.000479630873090529, 'n_batchsize': 168}. Best is trial 139 with value: 1201.6868896484375.
[I 2022-02-03 18:01:34,911] Trial 188 finished with value: 1181.7452392578125 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 1457, 'n_units_l1': 13, 'n_units_l2': 96, 'n_units_l3': 1111, 'lr': 0.0004471135856338432, 'n_batchsize': 148}. Best is trial 188 with value: 1181.7452392578125.
[I 2022-02-03 18:01:40,302] Trial 189 finished with value: 1217.2686767578125 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 1455, 'n_units_l1': 13, 'n_units_l2': 189, 'n_units_l3': 1053, 'lr': 0.0004493705656069996, 'n_batchsize': 147}. Best is trial 188 with value: 1181.7452392578125.
[I 2022-02-03 18:01:45,580] Trial 190 finished with value: 1218.7841796875 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 1245, 'n_units_l1': 15, 'n_units_l2': 106, 'n_units_l3': 1122, 'lr': 0.00043920010403116557, 'n_batchsize': 149}. Best is trial 188 with value: 1181.7452392578125.
[I 2022-02-03 18:01:50,949] Trial 191 finished with value: 1214.0380859375 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 1197, 'n_units_l1': 13, 'n_units_l2': 102, 'n_units_l3': 1076, 'lr': 0.00044547098642236453, 'n_batchsize': 146}. Best is trial 188 with value: 1181.7452392578125.
[I 2022-02-03 18:01:57,431] Trial 192 finished with value: 1215.8677978515625 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 1160, 'n_units_l1': 14, 'n_units_l2': 108, 'n_units_l3': 1148, 'lr': 0.0004631869293887525, 'n_batchsize': 134}. Best is trial 188 with value: 1181.7452392578125.
[I 2022-02-03 18:02:03,127] Trial 193 finished with value: 1217.10205078125 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 1070, 'n_units_l1': 13, 'n_units_l2': 87, 'n_units_l3': 1139, 'lr': 0.0004621976746156486, 'n_batchsize': 139}. Best is trial 188 with value: 1181.7452392578125.
[I 2022-02-03 18:02:08,975] Trial 194 finished with value: 1207.9610595703125 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 962, 'n_units_l1': 13, 'n_units_l2': 103, 'n_units_l3': 1050, 'lr': 0.0004584357859645516, 'n_batchsize': 138}. Best is trial 188 with value: 1181.7452392578125.
[I 2022-02-03 18:02:14,926] Trial 195 finished with value: 1209.2823486328125 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 751, 'n_units_l1': 13, 'n_units_l2': 87, 'n_units_l3': 1088, 'lr': 0.0004644518463366569, 'n_batchsize': 134}. Best is trial 188 with value: 1181.7452392578125.
[I 2022-02-03 18:02:20,887] Trial 196 finished with value: 1289.8154296875 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 759, 'n_units_l1': 13, 'n_units_l2': 101, 'n_units_l3': 1111, 'lr': 0.00046590589173269595, 'n_batchsize': 134}. Best is trial 188 with value: 1181.7452392578125.
[I 2022-02-03 18:02:26,329] Trial 197 finished with value: 1213.1817626953125 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 1082, 'n_units_l1': 13, 'n_units_l2': 101, 'n_units_l3': 1075, 'lr': 0.000433236335823582, 'n_batchsize': 142}. Best is trial 188 with value: 1181.7452392578125.
[I 2022-02-03 18:02:32,052] Trial 198 finished with value: 1269.4822998046875 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 955, 'n_units_l1': 14, 'n_units_l2': 101, 'n_units_l3': 1069, 'lr': 0.0006128958831915674, 'n_batchsize': 137}. Best is trial 188 with value: 1181.7452392578125.
[I 2022-02-03 18:02:37,387] Trial 199 finished with value: 1173.052978515625 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 1012, 'n_units_l1': 11, 'n_units_l2': 167, 'n_units_l3': 1308, 'lr': 0.00045720929224399954, 'n_batchsize': 144}. Best is trial 199 with value: 1173.052978515625.
[I 2022-02-03 18:02:42,741] Trial 200 finished with value: 1237.381591796875 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 816, 'n_units_l1': 10, 'n_units_l2': 162, 'n_units_l3': 1337, 'lr': 0.0006167772129359163, 'n_batchsize': 143}. Best is trial 199 with value: 1173.052978515625.
[I 2022-02-03 18:02:48,462] Trial 201 finished with value: 1295.1011962890625 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 1025, 'n_units_l1': 12, 'n_units_l2': 70, 'n_units_l3': 1035, 'lr': 0.0004511987382503511, 'n_batchsize': 131}. Best is trial 199 with value: 1173.052978515625.
[I 2022-02-03 18:02:54,365] Trial 202 finished with value: 1233.506591796875 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 902, 'n_units_l1': 14, 'n_units_l2': 106, 'n_units_l3': 1004, 'lr': 0.000470279194082329, 'n_batchsize': 145}. Best is trial 199 with value: 1173.052978515625.
[I 2022-02-03 18:02:59,823] Trial 203 finished with value: 1258.8336181640625 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 1061, 'n_units_l1': 11, 'n_units_l2': 83, 'n_units_l3': 1268, 'lr': 0.0004547449131711183, 'n_batchsize': 137}. Best is trial 199 with value: 1173.052978515625.
[I 2022-02-03 18:03:05,689] Trial 204 finished with value: 1360.443359375 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 1139, 'n_units_l1': 17, 'n_units_l2': 143, 'n_units_l3': 997, 'lr': 0.0006138993391355472, 'n_batchsize': 128}. Best is trial 199 with value: 1173.052978515625.
[I 2022-02-03 18:03:10,950] Trial 205 finished with value: 1187.3128662109375 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 970, 'n_units_l1': 14, 'n_units_l2': 86, 'n_units_l3': 1450, 'lr': 0.00046579185559649926, 'n_batchsize': 146}. Best is trial 199 with value: 1173.052978515625.
[I 2022-02-03 18:03:16,187] Trial 206 finished with value: 1219.6685791015625 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 939, 'n_units_l1': 15, 'n_units_l2': 87, 'n_units_l3': 1516, 'lr': 0.0004577038570727359, 'n_batchsize': 147}. Best is trial 199 with value: 1173.052978515625.
[I 2022-02-03 18:03:21,777] Trial 207 finished with value: 1308.3497314453125 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 654, 'n_units_l1': 12, 'n_units_l2': 119, 'n_units_l3': 1267, 'lr': 0.0006037789657307174, 'n_batchsize': 142}. Best is trial 199 with value: 1173.052978515625.
[I 2022-02-03 18:03:27,648] Trial 208 finished with value: 1232.408447265625 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 864, 'n_units_l1': 18, 'n_units_l2': 68, 'n_units_l3': 1594, 'lr': 0.0005269143039280909, 'n_batchsize': 149}. Best is trial 199 with value: 1173.052978515625.
[I 2022-02-03 18:03:34,059] Trial 209 finished with value: 1209.51416015625 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1016, 'n_units_l1': 13, 'n_units_l2': 92, 'n_units_l3': 981, 'lr': 0.00044804591264507426, 'n_batchsize': 136}. Best is trial 199 with value: 1173.052978515625.
[I 2022-02-03 18:03:40,326] Trial 210 finished with value: 1280.68017578125 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1058, 'n_units_l1': 13, 'n_units_l2': 92, 'n_units_l3': 883, 'lr': 0.0008459779868651774, 'n_batchsize': 141}. Best is trial 199 with value: 1173.052978515625.
[I 2022-02-03 18:03:46,546] Trial 211 finished with value: 1187.546630859375 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1156, 'n_units_l1': 10, 'n_units_l2': 145, 'n_units_l3': 1156, 'lr': 0.0004473498418616598, 'n_batchsize': 134}. Best is trial 199 with value: 1173.052978515625.
[I 2022-02-03 18:03:53,023] Trial 212 finished with value: 1241.3546142578125 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1157, 'n_units_l1': 11, 'n_units_l2': 148, 'n_units_l3': 1191, 'lr': 0.00043936847382121684, 'n_batchsize': 134}. Best is trial 199 with value: 1173.052978515625.
[I 2022-02-03 18:03:59,608] Trial 213 finished with value: 1280.3187255859375 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 974, 'n_units_l1': 10, 'n_units_l2': 113, 'n_units_l3': 1059, 'lr': 0.0007184740901840677, 'n_batchsize': 135}. Best is trial 199 with value: 1173.052978515625.
[I 2022-02-03 18:04:04,943] Trial 214 finished with value: 1272.494140625 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 749, 'n_units_l1': 14, 'n_units_l2': 203, 'n_units_l3': 859, 'lr': 0.0005570983561639873, 'n_batchsize': 149}. Best is trial 199 with value: 1173.052978515625.
[I 2022-02-03 18:04:10,446] Trial 215 finished with value: 1234.33203125 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 1041, 'n_units_l1': 13, 'n_units_l2': 93, 'n_units_l3': 1342, 'lr': 0.00044587132818083616, 'n_batchsize': 142}. Best is trial 199 with value: 1173.052978515625.
[I 2022-02-03 18:04:16,490] Trial 216 finished with value: 1219.419189453125 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 1212, 'n_units_l1': 15, 'n_units_l2': 74, 'n_units_l3': 1075, 'lr': 0.0005278378392748758, 'n_batchsize': 129}. Best is trial 199 with value: 1173.052978515625.
[I 2022-02-03 18:04:21,857] Trial 217 finished with value: 1242.6246337890625 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 813, 'n_units_l1': 12, 'n_units_l2': 172, 'n_units_l3': 1200, 'lr': 0.00028413801305332895, 'n_batchsize': 146}. Best is trial 199 with value: 1173.052978515625.
[I 2022-02-03 18:04:28,230] Trial 218 finished with value: 1236.608154296875 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 881, 'n_units_l1': 11, 'n_units_l2': 133, 'n_units_l3': 1848, 'lr': 0.0004261794676405287, 'n_batchsize': 137}. Best is trial 199 with value: 1173.052978515625.
[I 2022-02-03 18:04:33,592] Trial 219 finished with value: 1314.4404296875 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 1143, 'n_units_l1': 19, 'n_units_l2': 62, 'n_units_l3': 878, 'lr': 0.0006756790763816486, 'n_batchsize': 150}. Best is trial 199 with value: 1173.052978515625.
[I 2022-02-03 18:04:40,228] Trial 220 finished with value: 1245.5916748046875 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1009, 'n_units_l1': 16, 'n_units_l2': 110, 'n_units_l3': 1505, 'lr': 0.0005860519939744704, 'n_batchsize': 129}. Best is trial 199 with value: 1173.052978515625.
[I 2022-02-03 18:04:45,327] Trial 221 finished with value: 1225.351318359375 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 1213, 'n_units_l1': 13, 'n_units_l2': 94, 'n_units_l3': 964, 'lr': 0.0004175833352227891, 'n_batchsize': 153}. Best is trial 199 with value: 1173.052978515625.
[I 2022-02-03 18:04:50,854] Trial 222 finished with value: 1176.2486572265625 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 1112, 'n_units_l1': 15, 'n_units_l2': 79, 'n_units_l3': 1168, 'lr': 0.0003010562377796767, 'n_batchsize': 142}. Best is trial 199 with value: 1173.052978515625.
[I 2022-02-03 18:04:56,973] Trial 223 finished with value: 1240.7352294921875 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 1103, 'n_units_l1': 15, 'n_units_l2': 78, 'n_units_l3': 1193, 'lr': 0.0004976335091824936, 'n_batchsize': 139}. Best is trial 199 with value: 1173.052978515625.
[I 2022-02-03 18:05:02,621] Trial 224 finished with value: 1224.7587890625 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 918, 'n_units_l1': 12, 'n_units_l2': 124, 'n_units_l3': 738, 'lr': 0.00028886297171443214, 'n_batchsize': 141}. Best is trial 199 with value: 1173.052978515625.
[I 2022-02-03 18:05:08,056] Trial 225 finished with value: 1237.77587890625 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 1174, 'n_units_l1': 10, 'n_units_l2': 82, 'n_units_l3': 1381, 'lr': 0.0004249661257662102, 'n_batchsize': 148}. Best is trial 199 with value: 1173.052978515625.
[I 2022-02-03 18:05:13,241] Trial 226 finished with value: 1209.71337890625 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 1009, 'n_units_l1': 17, 'n_units_l2': 196, 'n_units_l3': 1104, 'lr': 0.0004904389547423396, 'n_batchsize': 157}. Best is trial 199 with value: 1173.052978515625.
[I 2022-02-03 18:05:19,330] Trial 227 finished with value: 1252.284423828125 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 1002, 'n_units_l1': 18, 'n_units_l2': 222, 'n_units_l3': 1107, 'lr': 0.0005193942434645279, 'n_batchsize': 135}. Best is trial 199 with value: 1173.052978515625.
[I 2022-02-03 18:05:24,681] Trial 228 finished with value: 1323.8363037109375 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 861, 'n_units_l1': 14, 'n_units_l2': 154, 'n_units_l3': 942, 'lr': 0.0006454939223193481, 'n_batchsize': 152}. Best is trial 199 with value: 1173.052978515625.
[I 2022-02-03 18:05:30,756] Trial 229 finished with value: 1348.1146240234375 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 275, 'n_units_l1': 16, 'n_units_l2': 184, 'n_units_l3': 1134, 'lr': 0.00042903708487643244, 'n_batchsize': 128}. Best is trial 199 with value: 1173.052978515625.
[I 2022-02-03 18:05:36,299] Trial 230 finished with value: 1222.0863037109375 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 1256, 'n_units_l1': 11, 'n_units_l2': 102, 'n_units_l3': 793, 'lr': 0.0007370099152733401, 'n_batchsize': 144}. Best is trial 199 with value: 1173.052978515625.
[I 2022-02-03 18:05:41,328] Trial 231 finished with value: 1204.7427978515625 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 1123, 'n_units_l1': 13, 'n_units_l2': 92, 'n_units_l3': 1293, 'lr': 0.0004019445457498129, 'n_batchsize': 157}. Best is trial 199 with value: 1173.052978515625.
[I 2022-02-03 18:05:46,551] Trial 232 finished with value: 1220.150146484375 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 1070, 'n_units_l1': 13, 'n_units_l2': 89, 'n_units_l3': 1186, 'lr': 0.00048535914450110866, 'n_batchsize': 157}. Best is trial 199 with value: 1173.052978515625.
[I 2022-02-03 18:05:52,292] Trial 233 finished with value: 1292.371337890625 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 934, 'n_units_l1': 17, 'n_units_l2': 109, 'n_units_l3': 1621, 'lr': 0.00056399279748743, 'n_batchsize': 139}. Best is trial 199 with value: 1173.052978515625.
[I 2022-02-03 18:05:58,021] Trial 234 finished with value: 1211.493896484375 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1260, 'n_units_l1': 15, 'n_units_l2': 97, 'n_units_l3': 1391, 'lr': 0.00040890995827503056, 'n_batchsize': 146}. Best is trial 199 with value: 1173.052978515625.
[I 2022-02-03 18:06:03,420] Trial 235 finished with value: 1188.903564453125 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1105, 'n_units_l1': 14, 'n_units_l2': 97, 'n_units_l3': 1381, 'lr': 0.00030300287307653226, 'n_batchsize': 156}. Best is trial 199 with value: 1173.052978515625.
[I 2022-02-03 18:06:09,112] Trial 236 finished with value: 1189.9998779296875 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1090, 'n_units_l1': 14, 'n_units_l2': 94, 'n_units_l3': 1353, 'lr': 0.0002892001222825438, 'n_batchsize': 153}. Best is trial 199 with value: 1173.052978515625.
[I 2022-02-03 18:06:14,753] Trial 237 finished with value: 1175.1549072265625 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1103, 'n_units_l1': 20, 'n_units_l2': 97, 'n_units_l3': 1461, 'lr': 0.00024858063905735156, 'n_batchsize': 157}. Best is trial 199 with value: 1173.052978515625.
[I 2022-02-03 18:06:20,360] Trial 238 finished with value: 1218.4583740234375 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1123, 'n_units_l1': 19, 'n_units_l2': 97, 'n_units_l3': 1971, 'lr': 0.000245017400261928, 'n_batchsize': 156}. Best is trial 199 with value: 1173.052978515625.
[I 2022-02-03 18:06:26,584] Trial 239 finished with value: 1206.5872802734375 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1006, 'n_units_l1': 17, 'n_units_l2': 122, 'n_units_l3': 1427, 'lr': 0.0002892142022054472, 'n_batchsize': 155}. Best is trial 199 with value: 1173.052978515625.
[I 2022-02-03 18:06:32,202] Trial 240 finished with value: 1229.8333740234375 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 943, 'n_units_l1': 21, 'n_units_l2': 131, 'n_units_l3': 1433, 'lr': 0.0002930149667936741, 'n_batchsize': 158}. Best is trial 199 with value: 1173.052978515625.
[I 2022-02-03 18:06:37,841] Trial 241 finished with value: 1181.133544921875 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 994, 'n_units_l1': 17, 'n_units_l2': 119, 'n_units_l3': 1372, 'lr': 0.0002476867315988684, 'n_batchsize': 158}. Best is trial 199 with value: 1173.052978515625.
[I 2022-02-03 18:06:43,630] Trial 242 finished with value: 1247.95556640625 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 841, 'n_units_l1': 17, 'n_units_l2': 78, 'n_units_l3': 1662, 'lr': 0.0002512082548911288, 'n_batchsize': 157}. Best is trial 199 with value: 1173.052978515625.
[I 2022-02-03 18:06:49,143] Trial 243 finished with value: 1236.3460693359375 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 718, 'n_units_l1': 19, 'n_units_l2': 120, 'n_units_l3': 1387, 'lr': 0.00025657808734430316, 'n_batchsize': 152}. Best is trial 199 with value: 1173.052978515625.
[I 2022-02-03 18:06:54,623] Trial 244 finished with value: 1227.14990234375 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1011, 'n_units_l1': 16, 'n_units_l2': 94, 'n_units_l3': 1383, 'lr': 0.0003101617070905269, 'n_batchsize': 161}. Best is trial 199 with value: 1173.052978515625.
[I 2022-02-03 18:07:00,541] Trial 245 finished with value: 1328.2879638671875 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 966, 'n_units_l1': 15, 'n_units_l2': 138, 'n_units_l3': 89, 'lr': 0.00029773219050506045, 'n_batchsize': 147}. Best is trial 199 with value: 1173.052978515625.
[I 2022-02-03 18:07:05,344] Trial 246 finished with value: 1193.5718994140625 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1073, 'n_units_l1': 12, 'n_units_l2': 114, 'n_units_l3': 1272, 'lr': 0.00022374690795568675, 'n_batchsize': 160}. Best is trial 199 with value: 1173.052978515625.
[I 2022-02-03 18:07:10,837] Trial 247 finished with value: 1225.105712890625 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 791, 'n_units_l1': 11, 'n_units_l2': 118, 'n_units_l3': 1270, 'lr': 0.0002415634721169896, 'n_batchsize': 159}. Best is trial 199 with value: 1173.052978515625.
[I 2022-02-03 18:07:16,443] Trial 248 finished with value: 1190.6175537109375 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1056, 'n_units_l1': 12, 'n_units_l2': 84, 'n_units_l3': 1518, 'lr': 0.0002009419574090668, 'n_batchsize': 152}. Best is trial 199 with value: 1173.052978515625.
[I 2022-02-03 18:07:22,206] Trial 249 finished with value: 1359.2080078125 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1029, 'n_units_l1': 12, 'n_units_l2': 53, 'n_units_l3': 1740, 'lr': 0.00013414841773097414, 'n_batchsize': 153}. Best is trial 199 with value: 1173.052978515625.
[I 2022-02-03 18:07:25,053] Trial 250 finished with value: 1457.618408203125 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 905, 'n_units_l1': 17, 'n_units_l2': 86, 'n_units_l3': 1512, 'lr': 0.00021574263389106114, 'n_batchsize': 368}. Best is trial 199 with value: 1173.052978515625.
[I 2022-02-03 18:07:30,415] Trial 251 finished with value: 1280.6407470703125 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1071, 'n_units_l1': 10, 'n_units_l2': 42, 'n_units_l3': 1299, 'lr': 0.00020561241660294562, 'n_batchsize': 163}. Best is trial 199 with value: 1173.052978515625.
[I 2022-02-03 18:07:36,047] Trial 252 finished with value: 1245.811279296875 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 977, 'n_units_l1': 20, 'n_units_l2': 76, 'n_units_l3': 1496, 'lr': 0.00018701922780140422, 'n_batchsize': 155}. Best is trial 199 with value: 1173.052978515625.
[I 2022-02-03 18:07:41,892] Trial 253 finished with value: 1194.670166015625 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1290, 'n_units_l1': 15, 'n_units_l2': 156, 'n_units_l3': 962, 'lr': 0.00026936485902871797, 'n_batchsize': 142}. Best is trial 199 with value: 1173.052978515625.
[I 2022-02-03 18:07:47,198] Trial 254 finished with value: 1355.28662109375 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 493, 'n_units_l1': 15, 'n_units_l2': 164, 'n_units_l3': 958, 'lr': 0.00015381313922286743, 'n_batchsize': 161}. Best is trial 199 with value: 1173.052978515625.
[I 2022-02-03 18:07:51,378] Trial 255 finished with value: 1467.39453125 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1330, 'n_units_l1': 17, 'n_units_l2': 63, 'n_units_l3': 1315, 'lr': 0.00017870765724351003, 'n_batchsize': 279}. Best is trial 199 with value: 1173.052978515625.
[I 2022-02-03 18:07:57,486] Trial 256 finished with value: 1171.244384765625 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1231, 'n_units_l1': 12, 'n_units_l2': 148, 'n_units_l3': 1718, 'lr': 0.00028101053239057715, 'n_batchsize': 151}. Best is trial 256 with value: 1171.244384765625.
[I 2022-02-03 18:08:03,482] Trial 257 finished with value: 1209.74609375 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1250, 'n_units_l1': 11, 'n_units_l2': 143, 'n_units_l3': 1687, 'lr': 0.0002400008375035668, 'n_batchsize': 147}. Best is trial 256 with value: 1171.244384765625.
[I 2022-02-03 18:08:09,602] Trial 258 finished with value: 1205.8140869140625 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1243, 'n_units_l1': 12, 'n_units_l2': 143, 'n_units_l3': 1747, 'lr': 0.0002421193953032312, 'n_batchsize': 146}. Best is trial 256 with value: 1171.244384765625.
[I 2022-02-03 18:08:15,983] Trial 259 finished with value: 1225.671875 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1177, 'n_units_l1': 11, 'n_units_l2': 145, 'n_units_l3': 1791, 'lr': 0.00023306034533564004, 'n_batchsize': 137}. Best is trial 256 with value: 1171.244384765625.
[I 2022-02-03 18:08:21,959] Trial 260 finished with value: 1232.1099853515625 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 864, 'n_units_l1': 12, 'n_units_l2': 134, 'n_units_l3': 1626, 'lr': 0.00027371628408238145, 'n_batchsize': 151}. Best is trial 256 with value: 1171.244384765625.
[I 2022-02-03 18:08:28,102] Trial 261 finished with value: 1442.4468994140625 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 149, 'n_units_l1': 10, 'n_units_l2': 125, 'n_units_l3': 1900, 'lr': 0.00019887096274071422, 'n_batchsize': 143}. Best is trial 256 with value: 1171.244384765625.
[I 2022-02-03 18:08:33,902] Trial 262 finished with value: 1260.465087890625 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1116, 'n_units_l1': 12, 'n_units_l2': 115, 'n_units_l3': 1704, 'lr': 0.00022565237155177886, 'n_batchsize': 151}. Best is trial 256 with value: 1171.244384765625.
[I 2022-02-03 18:08:39,983] Trial 263 finished with value: 1226.3936767578125 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1246, 'n_units_l1': 11, 'n_units_l2': 136, 'n_units_l3': 1507, 'lr': 0.00027097143004218524, 'n_batchsize': 136}. Best is trial 256 with value: 1171.244384765625.
[I 2022-02-03 18:08:45,613] Trial 264 finished with value: 1730.151611328125 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 987, 'n_units_l1': 14, 'n_units_l2': 153, 'n_units_l3': 15, 'lr': 0.00017384803025519336, 'n_batchsize': 155}. Best is trial 256 with value: 1171.244384765625.
[I 2022-02-03 18:08:48,117] Trial 265 finished with value: 1465.6751708984375 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1129, 'n_units_l1': 12, 'n_units_l2': 86, 'n_units_l3': 1220, 'lr': 0.00028723987557489894, 'n_batchsize': 484}. Best is trial 256 with value: 1171.244384765625.
[I 2022-02-03 18:08:54,796] Trial 266 finished with value: 1219.8480224609375 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 808, 'n_units_l1': 14, 'n_units_l2': 298, 'n_units_l3': 1262, 'lr': 0.0002354087891289783, 'n_batchsize': 128}. Best is trial 256 with value: 1171.244384765625.
[I 2022-02-03 18:09:00,688] Trial 267 finished with value: 1211.07421875 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1307, 'n_units_l1': 10, 'n_units_l2': 71, 'n_units_l3': 1494, 'lr': 0.00033037849255713634, 'n_batchsize': 143}. Best is trial 256 with value: 1171.244384765625.
[I 2022-02-03 18:09:06,486] Trial 268 finished with value: 1190.9598388671875 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1030, 'n_units_l1': 12, 'n_units_l2': 122, 'n_units_l3': 992, 'lr': 0.00031111430283638285, 'n_batchsize': 148}. Best is trial 256 with value: 1171.244384765625.
[I 2022-02-03 18:09:11,210] Trial 269 finished with value: 1219.2928466796875 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 918, 'n_units_l1': 13, 'n_units_l2': 112, 'n_units_l3': 995, 'lr': 0.0003038172777408766, 'n_batchsize': 160}. Best is trial 256 with value: 1171.244384765625.
[I 2022-02-03 18:09:18,188] Trial 270 finished with value: 1452.0084228515625 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 78, 'n_units_l1': 14, 'n_units_l2': 125, 'n_units_l3': 830, 'lr': 0.0002962768108146501, 'n_batchsize': 138}. Best is trial 256 with value: 1171.244384765625.
[I 2022-02-03 18:09:23,933] Trial 271 finished with value: 1238.2957763671875 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 627, 'n_units_l1': 15, 'n_units_l2': 176, 'n_units_l3': 898, 'lr': 0.0003314695482606075, 'n_batchsize': 152}. Best is trial 256 with value: 1171.244384765625.
[I 2022-02-03 18:09:29,917] Trial 272 finished with value: 1335.19287109375 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1019, 'n_units_l1': 12, 'n_units_l2': 208, 'n_units_l3': 1019, 'lr': 0.00011802258336958699, 'n_batchsize': 143}. Best is trial 256 with value: 1171.244384765625.
[I 2022-02-03 18:09:35,160] Trial 273 finished with value: 1170.5341796875 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1089, 'n_units_l1': 22, 'n_units_l2': 81, 'n_units_l3': 1187, 'lr': 0.00018991220625117107, 'n_batchsize': 161}. Best is trial 273 with value: 1170.5341796875.
[I 2022-02-03 18:09:41,555] Trial 274 finished with value: 1231.761962890625 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1123, 'n_units_l1': 21, 'n_units_l2': 81, 'n_units_l3': 1251, 'lr': 0.00015903812397087628, 'n_batchsize': 134}. Best is trial 273 with value: 1170.5341796875.
[I 2022-02-03 18:09:46,772] Trial 275 finished with value: 1260.037353515625 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 877, 'n_units_l1': 12, 'n_units_l2': 88, 'n_units_l3': 1390, 'lr': 0.0001997556389121662, 'n_batchsize': 164}. Best is trial 273 with value: 1170.5341796875.
[I 2022-02-03 18:09:52,564] Trial 276 finished with value: 1283.267822265625 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 362, 'n_units_l1': 19, 'n_units_l2': 94, 'n_units_l3': 1240, 'lr': 0.0002539312479751818, 'n_batchsize': 149}. Best is trial 273 with value: 1170.5341796875.
[I 2022-02-03 18:09:58,568] Trial 277 finished with value: 1251.9632568359375 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1341, 'n_units_l1': 14, 'n_units_l2': 109, 'n_units_l3': 781, 'lr': 0.0001827999340701375, 'n_batchsize': 143}. Best is trial 273 with value: 1170.5341796875.
[I 2022-02-03 18:10:00,993] Trial 278 finished with value: 1475.4459228515625 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1119, 'n_units_l1': 23, 'n_units_l2': 79, 'n_units_l3': 1136, 'lr': 0.00022274814055898916, 'n_batchsize': 511}. Best is trial 273 with value: 1170.5341796875.
[I 2022-02-03 18:10:02,150] Trial 279 finished with value: 1955.1226806640625 and parameters: {'n_layers': 4, 'n_epochs': 1, 'n_units_l0': 967, 'n_units_l1': 11, 'n_units_l2': 101, 'n_units_l3': 977, 'lr': 0.0003390170649754211, 'n_batchsize': 134}. Best is trial 273 with value: 1170.5341796875.
[I 2022-02-03 18:10:06,928] Trial 280 finished with value: 1240.1676025390625 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1214, 'n_units_l1': 16, 'n_units_l2': 68, 'n_units_l3': 1555, 'lr': 0.00015157094824820295, 'n_batchsize': 160}. Best is trial 273 with value: 1170.5341796875.
[I 2022-02-03 18:10:12,443] Trial 281 finished with value: 1328.2393798828125 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 192, 'n_units_l1': 13, 'n_units_l2': 116, 'n_units_l3': 1386, 'lr': 0.000249710999194522, 'n_batchsize': 151}. Best is trial 273 with value: 1170.5341796875.
[I 2022-02-03 18:10:18,431] Trial 282 finished with value: 1234.6761474609375 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 763, 'n_units_l1': 10, 'n_units_l2': 94, 'n_units_l3': 1982, 'lr': 0.00029530713797476857, 'n_batchsize': 143}. Best is trial 273 with value: 1170.5341796875.
[I 2022-02-03 18:10:23,726] Trial 283 finished with value: 1217.83740234375 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1053, 'n_units_l1': 14, 'n_units_l2': 86, 'n_units_l3': 1058, 'lr': 0.0003596505752358123, 'n_batchsize': 166}. Best is trial 273 with value: 1170.5341796875.
[I 2022-02-03 18:10:29,409] Trial 284 finished with value: 1200.69140625 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1327, 'n_units_l1': 12, 'n_units_l2': 129, 'n_units_l3': 871, 'lr': 0.0001919403512626605, 'n_batchsize': 154}. Best is trial 273 with value: 1170.5341796875.
[I 2022-02-03 18:10:35,661] Trial 285 finished with value: 1473.517822265625 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1352, 'n_units_l1': 12, 'n_units_l2': 123, 'n_units_l3': 1169, 'lr': 8.860935843914591e-05, 'n_batchsize': 152}. Best is trial 273 with value: 1170.5341796875.
[I 2022-02-03 18:10:41,989] Trial 286 finished with value: 1501.472412109375 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1176, 'n_units_l1': 11, 'n_units_l2': 145, 'n_units_l3': 237, 'lr': 0.0001335140689047404, 'n_batchsize': 135}. Best is trial 273 with value: 1170.5341796875.
[I 2022-02-03 18:10:47,709] Trial 287 finished with value: 1284.7607421875 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 915, 'n_units_l1': 13, 'n_units_l2': 108, 'n_units_l3': 858, 'lr': 0.00018074728244673642, 'n_batchsize': 158}. Best is trial 273 with value: 1170.5341796875.
[I 2022-02-03 18:10:53,677] Trial 288 finished with value: 1187.0281982421875 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1103, 'n_units_l1': 15, 'n_units_l2': 73, 'n_units_l3': 1310, 'lr': 0.00021101363485569996, 'n_batchsize': 145}. Best is trial 273 with value: 1170.5341796875.
[I 2022-02-03 18:10:55,330] Trial 289 finished with value: 2261.707275390625 and parameters: {'n_layers': 4, 'n_epochs': 2, 'n_units_l0': 22, 'n_units_l1': 16, 'n_units_l2': 74, 'n_units_l3': 1327, 'lr': 0.00019648911044083285, 'n_batchsize': 147}. Best is trial 273 with value: 1170.5341796875.
[I 2022-02-03 18:11:00,929] Trial 290 finished with value: 1219.114013671875 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1333, 'n_units_l1': 18, 'n_units_l2': 60, 'n_units_l3': 1450, 'lr': 0.00022557603436945624, 'n_batchsize': 154}. Best is trial 273 with value: 1170.5341796875.
[I 2022-02-03 18:11:07,040] Trial 291 finished with value: 1197.828125 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1115, 'n_units_l1': 14, 'n_units_l2': 127, 'n_units_l3': 1719, 'lr': 0.00015449404059366707, 'n_batchsize': 143}. Best is trial 273 with value: 1170.5341796875.
[I 2022-02-03 18:11:13,068] Trial 292 finished with value: 1219.76611328125 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1151, 'n_units_l1': 15, 'n_units_l2': 133, 'n_units_l3': 1687, 'lr': 0.00014982342182689903, 'n_batchsize': 143}. Best is trial 273 with value: 1170.5341796875.
[I 2022-02-03 18:11:19,045] Trial 293 finished with value: 1211.66455078125 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1067, 'n_units_l1': 11, 'n_units_l2': 154, 'n_units_l3': 1573, 'lr': 0.00014760535309113945, 'n_batchsize': 141}. Best is trial 273 with value: 1170.5341796875.
[I 2022-02-03 18:11:25,426] Trial 294 finished with value: 1238.8538818359375 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 852, 'n_units_l1': 14, 'n_units_l2': 123, 'n_units_l3': 1214, 'lr': 0.00017845367865416546, 'n_batchsize': 129}. Best is trial 273 with value: 1170.5341796875.
[I 2022-02-03 18:11:31,386] Trial 295 finished with value: 1209.0797119140625 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1244, 'n_units_l1': 12, 'n_units_l2': 67, 'n_units_l3': 1420, 'lr': 0.0002140044571946498, 'n_batchsize': 149}. Best is trial 273 with value: 1170.5341796875.
[I 2022-02-03 18:11:37,058] Trial 296 finished with value: 1245.7176513671875 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1272, 'n_units_l1': 10, 'n_units_l2': 71, 'n_units_l3': 1664, 'lr': 0.00019736368073568612, 'n_batchsize': 154}. Best is trial 273 with value: 1170.5341796875.
[I 2022-02-03 18:11:42,585] Trial 297 finished with value: 1219.3529052734375 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1119, 'n_units_l1': 12, 'n_units_l2': 51, 'n_units_l3': 1770, 'lr': 0.00021662007184314205, 'n_batchsize': 161}. Best is trial 273 with value: 1170.5341796875.
[I 2022-02-03 18:11:48,548] Trial 298 finished with value: 1253.8458251953125 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1218, 'n_units_l1': 16, 'n_units_l2': 67, 'n_units_l3': 1446, 'lr': 0.00015986080045216408, 'n_batchsize': 148}. Best is trial 273 with value: 1170.5341796875.
[I 2022-02-03 18:11:54,289] Trial 299 finished with value: 1251.474365234375 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 978, 'n_units_l1': 15, 'n_units_l2': 10, 'n_units_l3': 1383, 'lr': 0.00025518462240959136, 'n_batchsize': 149}. Best is trial 273 with value: 1170.5341796875.
[I 2022-02-03 18:11:58,949] Trial 300 finished with value: 1194.30712890625 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1330, 'n_units_l1': 12, 'n_units_l2': 56, 'n_units_l3': 1840, 'lr': 0.00026160126106400763, 'n_batchsize': 160}. Best is trial 273 with value: 1170.5341796875.
[I 2022-02-03 18:12:05,143] Trial 301 finished with value: 1392.2220458984375 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1399, 'n_units_l1': 18, 'n_units_l2': 57, 'n_units_l3': 1943, 'lr': 0.00011079726633684702, 'n_batchsize': 161}. Best is trial 273 with value: 1170.5341796875.
[I 2022-02-03 18:12:10,935] Trial 302 finished with value: 1389.2320556640625 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1067, 'n_units_l1': 10, 'n_units_l2': 47, 'n_units_l3': 1773, 'lr': 0.00013336712876444007, 'n_batchsize': 156}. Best is trial 273 with value: 1170.5341796875.
[I 2022-02-03 18:12:16,385] Trial 303 finished with value: 1354.40087890625 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1152, 'n_units_l1': 14, 'n_units_l2': 41, 'n_units_l3': 125, 'lr': 0.0002622750173514001, 'n_batchsize': 167}. Best is trial 273 with value: 1170.5341796875.
[I 2022-02-03 18:12:22,627] Trial 304 finished with value: 1227.2286376953125 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 937, 'n_units_l1': 11, 'n_units_l2': 169, 'n_units_l3': 1575, 'lr': 0.00017537743517316362, 'n_batchsize': 141}. Best is trial 273 with value: 1170.5341796875.
[I 2022-02-03 18:12:28,198] Trial 305 finished with value: 1178.1956787109375 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1372, 'n_units_l1': 12, 'n_units_l2': 135, 'n_units_l3': 1251, 'lr': 0.0002646975972727699, 'n_batchsize': 159}. Best is trial 273 with value: 1170.5341796875.
[I 2022-02-03 18:12:33,525] Trial 306 finished with value: 1194.110595703125 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1347, 'n_units_l1': 12, 'n_units_l2': 132, 'n_units_l3': 1273, 'lr': 0.0002596365114555533, 'n_batchsize': 167}. Best is trial 273 with value: 1170.5341796875.
[I 2022-02-03 18:12:38,994] Trial 307 finished with value: 1232.962890625 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1415, 'n_units_l1': 12, 'n_units_l2': 139, 'n_units_l3': 1243, 'lr': 0.00023499057868925586, 'n_batchsize': 168}. Best is trial 273 with value: 1170.5341796875.
[I 2022-02-03 18:12:44,592] Trial 308 finished with value: 1327.0362548828125 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1354, 'n_units_l1': 11, 'n_units_l2': 17, 'n_units_l3': 1166, 'lr': 0.00019683304526986543, 'n_batchsize': 163}. Best is trial 273 with value: 1170.5341796875.
[I 2022-02-03 18:12:47,842] Trial 309 finished with value: 1226.5394287109375 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1280, 'n_units_l1': 13, 'n_units_l2': 161, 'n_units_l3': 1300, 'lr': 0.000267188937312985, 'n_batchsize': 318}. Best is trial 273 with value: 1170.5341796875.
[I 2022-02-03 18:12:52,700] Trial 310 finished with value: 1203.74951171875 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1439, 'n_units_l1': 10, 'n_units_l2': 135, 'n_units_l3': 1578, 'lr': 0.0003104235580369261, 'n_batchsize': 160}. Best is trial 273 with value: 1170.5341796875.
[I 2022-02-03 18:12:57,825] Trial 311 finished with value: 1193.3585205078125 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1495, 'n_units_l1': 11, 'n_units_l2': 129, 'n_units_l3': 1601, 'lr': 0.0003330662194226511, 'n_batchsize': 171}. Best is trial 273 with value: 1170.5341796875.
[I 2022-02-03 18:13:03,053] Trial 312 finished with value: 1196.3900146484375 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1477, 'n_units_l1': 11, 'n_units_l2': 133, 'n_units_l3': 1481, 'lr': 0.0003275859677310013, 'n_batchsize': 170}. Best is trial 273 with value: 1170.5341796875.
[I 2022-02-03 18:13:05,621] Trial 313 finished with value: 1296.929443359375 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1424, 'n_units_l1': 11, 'n_units_l2': 117, 'n_units_l3': 1108, 'lr': 0.0003433006804785143, 'n_batchsize': 453}. Best is trial 273 with value: 1170.5341796875.
[I 2022-02-03 18:13:10,702] Trial 314 finished with value: 1226.3680419921875 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1490, 'n_units_l1': 10, 'n_units_l2': 153, 'n_units_l3': 1529, 'lr': 0.0002720296900277501, 'n_batchsize': 171}. Best is trial 273 with value: 1170.5341796875.
[I 2022-02-03 18:13:15,733] Trial 315 finished with value: 1802.8106689453125 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1320, 'n_units_l1': 15, 'n_units_l2': 135, 'n_units_l3': 35, 'lr': 7.355341726312739e-05, 'n_batchsize': 172}. Best is trial 273 with value: 1170.5341796875.
[I 2022-02-03 18:13:21,715] Trial 316 finished with value: 1244.733154296875 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1503, 'n_units_l1': 11, 'n_units_l2': 122, 'n_units_l3': 679, 'lr': 0.00021848682901730377, 'n_batchsize': 166}. Best is trial 273 with value: 1170.5341796875.
[I 2022-02-03 18:13:26,953] Trial 317 finished with value: 1239.8135986328125 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1259, 'n_units_l1': 12, 'n_units_l2': 110, 'n_units_l3': 392, 'lr': 0.0003270314262258389, 'n_batchsize': 163}. Best is trial 273 with value: 1170.5341796875.
[I 2022-02-03 18:13:32,339] Trial 318 finished with value: 1460.6795654296875 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1483, 'n_units_l1': 14, 'n_units_l2': 173, 'n_units_l3': 1313, 'lr': 0.015409610974958162, 'n_batchsize': 171}. Best is trial 273 with value: 1170.5341796875.
[I 2022-02-03 18:13:34,970] Trial 319 finished with value: 1687.1326904296875 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 36, 'n_units_l1': 10, 'n_units_l2': 129, 'n_units_l3': 1892, 'lr': 0.00017830789369441887, 'n_batchsize': 437}. Best is trial 273 with value: 1170.5341796875.
[I 2022-02-03 18:13:40,854] Trial 320 finished with value: 1238.144775390625 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1327, 'n_units_l1': 11, 'n_units_l2': 239, 'n_units_l3': 1174, 'lr': 0.0002682848730097338, 'n_batchsize': 153}. Best is trial 273 with value: 1170.5341796875.
[I 2022-02-03 18:13:46,324] Trial 321 finished with value: 1159.6004638671875 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1167, 'n_units_l1': 14, 'n_units_l2': 111, 'n_units_l3': 1502, 'lr': 0.00034296931053536195, 'n_batchsize': 161}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:13:51,912] Trial 322 finished with value: 1179.3189697265625 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1166, 'n_units_l1': 14, 'n_units_l2': 109, 'n_units_l3': 1593, 'lr': 0.00032846051680373596, 'n_batchsize': 159}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:13:57,532] Trial 323 finished with value: 1210.266845703125 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1159, 'n_units_l1': 13, 'n_units_l2': 108, 'n_units_l3': 1618, 'lr': 0.00021590117514486232, 'n_batchsize': 158}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:14:02,904] Trial 324 finished with value: 1217.7554931640625 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1083, 'n_units_l1': 15, 'n_units_l2': 159, 'n_units_l3': 1473, 'lr': 0.0002921016265477279, 'n_batchsize': 164}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:14:08,658] Trial 325 finished with value: 1366.7581787109375 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1171, 'n_units_l1': 1257, 'n_units_l2': 79, 'n_units_l3': 1772, 'lr': 0.00013471193035818606, 'n_batchsize': 153}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:14:14,042] Trial 326 finished with value: 1175.326904296875 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1058, 'n_units_l1': 12, 'n_units_l2': 100, 'n_units_l3': 1992, 'lr': 0.00032529144508119473, 'n_batchsize': 162}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:14:19,369] Trial 327 finished with value: 1212.1856689453125 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1035, 'n_units_l1': 14, 'n_units_l2': 103, 'n_units_l3': 1933, 'lr': 0.0003406171667073902, 'n_batchsize': 170}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:14:24,626] Trial 328 finished with value: 1246.2322998046875 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 895, 'n_units_l1': 13, 'n_units_l2': 115, 'n_units_l3': 1630, 'lr': 0.0003123333128491566, 'n_batchsize': 164}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:14:30,168] Trial 329 finished with value: 1213.9462890625 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1062, 'n_units_l1': 16, 'n_units_l2': 81, 'n_units_l3': 1384, 'lr': 0.0002590234328220209, 'n_batchsize': 159}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:14:35,217] Trial 330 finished with value: 1166.306884765625 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1184, 'n_units_l1': 11, 'n_units_l2': 99, 'n_units_l3': 1811, 'lr': 0.0003552759621792295, 'n_batchsize': 173}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:14:40,410] Trial 331 finished with value: 1194.825439453125 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1207, 'n_units_l1': 11, 'n_units_l2': 97, 'n_units_l3': 1971, 'lr': 0.0003498625808407813, 'n_batchsize': 173}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:14:46,130] Trial 332 finished with value: 1213.1962890625 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1164, 'n_units_l1': 12, 'n_units_l2': 96, 'n_units_l3': 1905, 'lr': 0.0003519147129924717, 'n_batchsize': 176}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:14:51,265] Trial 333 finished with value: 1221.35302734375 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1020, 'n_units_l1': 10, 'n_units_l2': 98, 'n_units_l3': 1957, 'lr': 0.0003624607933854807, 'n_batchsize': 173}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:14:56,848] Trial 334 finished with value: 1272.1005859375 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 950, 'n_units_l1': 11, 'n_units_l2': 102, 'n_units_l3': 1706, 'lr': 0.0002625337197395097, 'n_batchsize': 164}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:15:02,409] Trial 335 finished with value: 1185.6474609375 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 833, 'n_units_l1': 12, 'n_units_l2': 82, 'n_units_l3': 1284, 'lr': 0.00036102030251598913, 'n_batchsize': 159}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:15:08,213] Trial 336 finished with value: 1213.4024658203125 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 906, 'n_units_l1': 13, 'n_units_l2': 80, 'n_units_l3': 1296, 'lr': 0.00029413298316331894, 'n_batchsize': 150}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:15:12,880] Trial 337 finished with value: 1227.9415283203125 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 791, 'n_units_l1': 12, 'n_units_l2': 84, 'n_units_l3': 1217, 'lr': 0.00023809966491439475, 'n_batchsize': 160}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:15:18,389] Trial 338 finished with value: 1232.98681640625 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 825, 'n_units_l1': 15, 'n_units_l2': 73, 'n_units_l3': 1510, 'lr': 0.0003658025401567786, 'n_batchsize': 155}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:15:20,658] Trial 339 finished with value: 1535.115234375 and parameters: {'n_layers': 4, 'n_epochs': 3, 'n_units_l0': 1037, 'n_units_l1': 16, 'n_units_l2': 85, 'n_units_l3': 1324, 'lr': 0.00029044799697800546, 'n_batchsize': 148}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:15:26,080] Trial 340 finished with value: 1223.814453125 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1209, 'n_units_l1': 13, 'n_units_l2': 112, 'n_units_l3': 1116, 'lr': 0.00022457206005020342, 'n_batchsize': 162}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:15:31,712] Trial 341 finished with value: 1205.645751953125 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 685, 'n_units_l1': 12, 'n_units_l2': 90, 'n_units_l3': 1424, 'lr': 0.00037094871919129843, 'n_batchsize': 156}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:15:37,474] Trial 342 finished with value: 1283.775634765625 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 963, 'n_units_l1': 536, 'n_units_l2': 75, 'n_units_l3': 1569, 'lr': 0.0002776779060127923, 'n_batchsize': 147}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:15:42,901] Trial 343 finished with value: 1190.721923828125 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1085, 'n_units_l1': 14, 'n_units_l2': 108, 'n_units_l3': 1065, 'lr': 0.00038215350981287886, 'n_batchsize': 165}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:15:48,161] Trial 344 finished with value: 2079.6357421875 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 847, 'n_units_l1': 10, 'n_units_l2': 106, 'n_units_l3': 1236, 'lr': 0.0645084671691462, 'n_batchsize': 167}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:15:53,209] Trial 345 finished with value: 1243.122314453125 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1082, 'n_units_l1': 14, 'n_units_l2': 95, 'n_units_l3': 1117, 'lr': 0.0003871935893452132, 'n_batchsize': 178}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:15:55,899] Trial 346 finished with value: 1359.26611328125 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1049, 'n_units_l1': 12, 'n_units_l2': 111, 'n_units_l3': 1372, 'lr': 0.0003193304137409146, 'n_batchsize': 413}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:16:01,394] Trial 347 finished with value: 1202.2586669921875 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 930, 'n_units_l1': 13, 'n_units_l2': 60, 'n_units_l3': 1692, 'lr': 0.00037533673511043453, 'n_batchsize': 162}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:16:07,281] Trial 348 finished with value: 1200.0640869140625 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1125, 'n_units_l1': 11, 'n_units_l2': 89, 'n_units_l3': 1526, 'lr': 0.00023182398215926297, 'n_batchsize': 166}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:16:13,013] Trial 349 finished with value: 2076.8681640625 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 983, 'n_units_l1': 14, 'n_units_l2': 103, 'n_units_l3': 1055, 'lr': 1.9045036930336303e-05, 'n_batchsize': 156}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:16:18,018] Trial 350 finished with value: 1177.5 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1184, 'n_units_l1': 10, 'n_units_l2': 121, 'n_units_l3': 1277, 'lr': 0.0003098202849326427, 'n_batchsize': 174}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:16:22,938] Trial 351 finished with value: 1177.419189453125 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 879, 'n_units_l1': 10, 'n_units_l2': 119, 'n_units_l3': 1249, 'lr': 0.0003887987892325957, 'n_batchsize': 183}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:16:27,856] Trial 352 finished with value: 1220.034423828125 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 843, 'n_units_l1': 10, 'n_units_l2': 115, 'n_units_l3': 1152, 'lr': 0.0003880531166225672, 'n_batchsize': 182}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:16:32,821] Trial 353 finished with value: 1237.264404296875 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 764, 'n_units_l1': 10, 'n_units_l2': 120, 'n_units_l3': 1335, 'lr': 0.00037110373524655945, 'n_batchsize': 186}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:16:37,958] Trial 354 finished with value: 1264.246826171875 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 915, 'n_units_l1': 10, 'n_units_l2': 88, 'n_units_l3': 1040, 'lr': 0.0003181634681587067, 'n_batchsize': 176}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:16:43,145] Trial 355 finished with value: 1227.5108642578125 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1006, 'n_units_l1': 17, 'n_units_l2': 104, 'n_units_l3': 1226, 'lr': 0.00038185055822015335, 'n_batchsize': 173}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:16:48,830] Trial 356 finished with value: 1184.05029296875 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 871, 'n_units_l1': 11, 'n_units_l2': 78, 'n_units_l3': 1464, 'lr': 0.0003337279685041818, 'n_batchsize': 151}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:16:54,581] Trial 357 finished with value: 1255.5147705078125 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 669, 'n_units_l1': 11, 'n_units_l2': 72, 'n_units_l3': 1434, 'lr': 0.0003943018071153398, 'n_batchsize': 150}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:17:00,178] Trial 358 finished with value: 1228.2138671875 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 717, 'n_units_l1': 10, 'n_units_l2': 80, 'n_units_l3': 1544, 'lr': 0.00031935083152458414, 'n_batchsize': 149}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:17:06,114] Trial 359 finished with value: 1238.3795166015625 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 587, 'n_units_l1': 20, 'n_units_l2': 64, 'n_units_l3': 1393, 'lr': 0.00030659325694340944, 'n_batchsize': 141}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:17:11,660] Trial 360 finished with value: 1235.2012939453125 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 848, 'n_units_l1': 11, 'n_units_l2': 93, 'n_units_l3': 1676, 'lr': 0.0003843625925196723, 'n_batchsize': 154}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:17:16,783] Trial 361 finished with value: 1398.4097900390625 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 118, 'n_units_l1': 14, 'n_units_l2': 81, 'n_units_l3': 1178, 'lr': 0.00030198532374465023, 'n_batchsize': 171}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:17:19,774] Trial 362 finished with value: 1238.450927734375 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 886, 'n_units_l1': 16, 'n_units_l2': 75, 'n_units_l3': 1466, 'lr': 0.000406523690323327, 'n_batchsize': 347}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:17:26,329] Trial 363 finished with value: 1189.659423828125 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 810, 'n_units_l1': 10, 'n_units_l2': 94, 'n_units_l3': 1307, 'lr': 0.0003307902901221502, 'n_batchsize': 146}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:17:32,652] Trial 364 finished with value: 1237.6199951171875 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 797, 'n_units_l1': 10, 'n_units_l2': 91, 'n_units_l3': 1026, 'lr': 0.0005274089719501193, 'n_batchsize': 137}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:17:36,198] Trial 365 finished with value: 1251.2626953125 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 921, 'n_units_l1': 13, 'n_units_l2': 96, 'n_units_l3': 1288, 'lr': 0.000303050570954075, 'n_batchsize': 290}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:17:42,383] Trial 366 finished with value: 1254.822998046875 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 743, 'n_units_l1': 18, 'n_units_l2': 83, 'n_units_l3': 1188, 'lr': 0.00037056063956137904, 'n_batchsize': 147}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:17:48,309] Trial 367 finished with value: 1281.1268310546875 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 811, 'n_units_l1': 11, 'n_units_l2': 102, 'n_units_l3': 1358, 'lr': 0.00025167267672597727, 'n_batchsize': 144}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:17:55,002] Trial 368 finished with value: 1323.1285400390625 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 999, 'n_units_l1': 203, 'n_units_l2': 74, 'n_units_l3': 1090, 'lr': 0.00019696092273329023, 'n_batchsize': 129}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:18:00,710] Trial 369 finished with value: 1399.4791259765625 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 883, 'n_units_l1': 1758, 'n_units_l2': 88, 'n_units_l3': 1288, 'lr': 0.0004181843609827532, 'n_batchsize': 154}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:18:06,877] Trial 370 finished with value: 1234.9002685546875 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1148, 'n_units_l1': 15, 'n_units_l2': 68, 'n_units_l3': 938, 'lr': 0.000310497930709757, 'n_batchsize': 137}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:18:12,543] Trial 371 finished with value: 1207.167724609375 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1014, 'n_units_l1': 10, 'n_units_l2': 110, 'n_units_l3': 1776, 'lr': 0.0005468957144263511, 'n_batchsize': 148}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:18:17,956] Trial 372 finished with value: 1181.471923828125 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1122, 'n_units_l1': 12, 'n_units_l2': 98, 'n_units_l3': 1480, 'lr': 0.0002486198377245129, 'n_batchsize': 157}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:18:22,595] Trial 373 finished with value: 1160.1307373046875 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1166, 'n_units_l1': 13, 'n_units_l2': 97, 'n_units_l3': 1550, 'lr': 0.00021786670467415333, 'n_batchsize': 160}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:18:28,214] Trial 374 finished with value: 1321.74560546875 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1258, 'n_units_l1': 13, 'n_units_l2': 83, 'n_units_l3': 161, 'lr': 0.00019468862683549848, 'n_batchsize': 158}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:18:33,685] Trial 375 finished with value: 1253.8031005859375 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1165, 'n_units_l1': 11, 'n_units_l2': 95, 'n_units_l3': 1504, 'lr': 0.0001717462026511275, 'n_batchsize': 156}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:18:35,348] Trial 376 finished with value: 2567.3466796875 and parameters: {'n_layers': 4, 'n_epochs': 2, 'n_units_l0': 11, 'n_units_l1': 12, 'n_units_l2': 77, 'n_units_l3': 516, 'lr': 0.0002177462850121436, 'n_batchsize': 143}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:18:40,736] Trial 377 finished with value: 1430.7027587890625 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 52, 'n_units_l1': 10, 'n_units_l2': 90, 'n_units_l3': 1695, 'lr': 0.00024883843295975836, 'n_batchsize': 161}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:18:46,382] Trial 378 finished with value: 1216.5281982421875 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 931, 'n_units_l1': 16, 'n_units_l2': 100, 'n_units_l3': 1564, 'lr': 0.00024864725248649965, 'n_batchsize': 150}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:18:53,489] Trial 379 finished with value: 1200.5347900390625 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1145, 'n_units_l1': 12, 'n_units_l2': 81, 'n_units_l3': 1417, 'lr': 0.00019955539017885442, 'n_batchsize': 134}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:18:59,230] Trial 380 finished with value: 1308.3660888671875 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 759, 'n_units_l1': 13, 'n_units_l2': 96, 'n_units_l3': 1741, 'lr': 0.00016404584342742128, 'n_batchsize': 155}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:19:04,317] Trial 381 finished with value: 1210.1002197265625 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 1194, 'n_units_l1': 11, 'n_units_l2': 66, 'n_units_l3': 1414, 'lr': 0.0002629504705654041, 'n_batchsize': 162}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:19:10,733] Trial 382 finished with value: 1606.6724853515625 and parameters: {'n_layers': 5, 'n_epochs': 10, 'n_units_l0': 956, 'n_units_l1': 19, 'n_units_l2': 85, 'n_units_l3': 69, 'n_units_l4': 34, 'lr': 0.0002220155379827473, 'n_batchsize': 141}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:19:16,465] Trial 383 finished with value: 1182.81787109375 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1078, 'n_units_l1': 15, 'n_units_l2': 115, 'n_units_l3': 1595, 'lr': 0.0002980835039001317, 'n_batchsize': 151}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:19:21,926] Trial 384 finished with value: 1181.2821044921875 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1236, 'n_units_l1': 17, 'n_units_l2': 116, 'n_units_l3': 1632, 'lr': 0.00029317315240448777, 'n_batchsize': 166}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:19:27,152] Trial 385 finished with value: 1190.337158203125 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1217, 'n_units_l1': 21, 'n_units_l2': 145, 'n_units_l3': 1986, 'lr': 0.0003006360021320172, 'n_batchsize': 179}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:19:32,019] Trial 386 finished with value: 1171.2724609375 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 1262, 'n_units_l1': 18, 'n_units_l2': 117, 'n_units_l3': 1650, 'lr': 0.00033408962693277233, 'n_batchsize': 167}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:19:36,685] Trial 387 finished with value: 1199.521728515625 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 1256, 'n_units_l1': 18, 'n_units_l2': 119, 'n_units_l3': 1784, 'lr': 0.0004122178180923701, 'n_batchsize': 177}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:19:41,547] Trial 388 finished with value: 1232.4376220703125 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 1315, 'n_units_l1': 18, 'n_units_l2': 145, 'n_units_l3': 1624, 'lr': 0.0002732476879417587, 'n_batchsize': 169}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:19:46,519] Trial 389 finished with value: 1258.12060546875 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 1187, 'n_units_l1': 22, 'n_units_l2': 179, 'n_units_l3': 1797, 'lr': 0.0003535411392395585, 'n_batchsize': 168}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:19:50,773] Trial 390 finished with value: 1208.431396484375 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 1131, 'n_units_l1': 16, 'n_units_l2': 119, 'n_units_l3': 1594, 'lr': 0.00025923735500919554, 'n_batchsize': 187}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:19:55,639] Trial 391 finished with value: 1436.08935546875 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 81, 'n_units_l1': 20, 'n_units_l2': 120, 'n_units_l3': 1569, 'lr': 0.00052250815254069, 'n_batchsize': 167}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:19:59,340] Trial 392 finished with value: 1669.273193359375 and parameters: {'n_layers': 1, 'n_epochs': 9, 'n_units_l0': 1304, 'lr': 0.0004039301358348939, 'n_batchsize': 177}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:20:04,324] Trial 393 finished with value: 1199.9256591796875 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 1046, 'n_units_l1': 17, 'n_units_l2': 108, 'n_units_l3': 1815, 'lr': 0.0003237745576351656, 'n_batchsize': 163}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:20:09,028] Trial 394 finished with value: 1745.3260498046875 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1187, 'n_units_l1': 15, 'n_units_l2': 147, 'n_units_l3': 1451, 'lr': 0.039590601183568594, 'n_batchsize': 160}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:20:14,921] Trial 395 finished with value: 1222.880615234375 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 986, 'n_units_l1': 17, 'n_units_l2': 110, 'n_units_l3': 1599, 'lr': 0.00024001469115345728, 'n_batchsize': 174}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:20:20,623] Trial 396 finished with value: 1198.8824462890625 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1355, 'n_units_l1': 15, 'n_units_l2': 123, 'n_units_l3': 1222, 'lr': 0.0002855838258835677, 'n_batchsize': 158}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:20:25,728] Trial 397 finished with value: 1376.36181640625 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 231, 'n_units_l1': 19, 'n_units_l2': 105, 'n_units_l3': 1984, 'lr': 0.00039896034510541784, 'n_batchsize': 166}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:20:30,750] Trial 398 finished with value: 1248.9615478515625 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1086, 'n_units_l1': 16, 'n_units_l2': 138, 'n_units_l3': 1421, 'lr': 0.00021273297535882188, 'n_batchsize': 184}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:20:36,101] Trial 399 finished with value: 1201.191650390625 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1233, 'n_units_l1': 14, 'n_units_l2': 114, 'n_units_l3': 1729, 'lr': 0.0003280754740865787, 'n_batchsize': 173}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:20:41,821] Trial 400 finished with value: 2738.246337890625 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 940, 'n_units_l1': 14, 'n_units_l2': 132, 'n_units_l3': 11, 'lr': 0.00047051504978064615, 'n_batchsize': 155}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:20:47,419] Trial 401 finished with value: 1329.25634765625 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1079, 'n_units_l1': 683, 'n_units_l2': 194, 'n_units_l3': 1312, 'lr': 0.0006152083846199557, 'n_batchsize': 164}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:20:52,590] Trial 402 finished with value: 1169.39990234375 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 1391, 'n_units_l1': 22, 'n_units_l2': 102, 'n_units_l3': 1499, 'lr': 0.00023094618577255608, 'n_batchsize': 151}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:20:56,169] Trial 403 finished with value: 1317.6337890625 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 1438, 'n_units_l1': 21, 'n_units_l2': 161, 'n_units_l3': 1600, 'lr': 0.00018321761847217647, 'n_batchsize': 253}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:21:01,790] Trial 404 finished with value: 1254.193359375 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 1386, 'n_units_l1': 22, 'n_units_l2': 104, 'n_units_l3': 279, 'lr': 0.00022746512772948524, 'n_batchsize': 139}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:21:07,115] Trial 405 finished with value: 1310.4029541015625 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 389, 'n_units_l1': 20, 'n_units_l2': 122, 'n_units_l3': 2000, 'lr': 0.00025402439906918356, 'n_batchsize': 147}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:21:12,458] Trial 406 finished with value: 1244.371826171875 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 1280, 'n_units_l1': 24, 'n_units_l2': 104, 'n_units_l3': 1479, 'lr': 0.0001746902542681277, 'n_batchsize': 151}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:21:18,315] Trial 407 finished with value: 1257.4503173828125 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 1531, 'n_units_l1': 19, 'n_units_l2': 146, 'n_units_l3': 1194, 'lr': 0.0003463360050258783, 'n_batchsize': 128}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:21:23,181] Trial 408 finished with value: 1214.0516357421875 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 1233, 'n_units_l1': 23, 'n_units_l2': 115, 'n_units_l3': 1764, 'lr': 0.000221467529876976, 'n_batchsize': 159}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:21:28,649] Trial 409 finished with value: 1611.6031494140625 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 19, 'n_units_l1': 17, 'n_units_l2': 72, 'n_units_l3': 1347, 'lr': 0.00027567978335392787, 'n_batchsize': 140}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:21:31,142] Trial 410 finished with value: 1727.617919921875 and parameters: {'n_layers': 4, 'n_epochs': 3, 'n_units_l0': 1390, 'n_units_l1': 13, 'n_units_l2': 125, 'n_units_l3': 1534, 'lr': 0.00014349080233618146, 'n_batchsize': 181}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:21:36,271] Trial 411 finished with value: 1264.894775390625 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1142, 'n_units_l1': 15, 'n_units_l2': 91, 'n_units_l3': 1150, 'lr': 0.000402209486026777, 'n_batchsize': 170}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:21:42,058] Trial 412 finished with value: 1647.7548828125 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 880, 'n_units_l1': 12, 'n_units_l2': 101, 'n_units_l3': 39, 'lr': 0.000204849857717568, 'n_batchsize': 150}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:21:46,909] Trial 413 finished with value: 1238.8277587890625 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 1011, 'n_units_l1': 18, 'n_units_l2': 135, 'n_units_l3': 1669, 'lr': 0.0005093061566587388, 'n_batchsize': 163}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:21:49,792] Trial 414 finished with value: 1626.8594970703125 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1264, 'n_units_l1': 11, 'n_units_l2': 75, 'n_units_l3': 1276, 'lr': 0.00011257732640494005, 'n_batchsize': 376}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:21:56,215] Trial 415 finished with value: 1611.344970703125 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 980, 'n_units_l1': 16, 'n_units_l2': 109, 'n_units_l3': 1446, 'lr': 4.813499578849131e-05, 'n_batchsize': 135}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:22:02,351] Trial 416 finished with value: 1202.9766845703125 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1568, 'n_units_l1': 13, 'n_units_l2': 167, 'n_units_l3': 1801, 'lr': 0.00029710345760971553, 'n_batchsize': 145}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:22:08,106] Trial 417 finished with value: 1242.740234375 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1146, 'n_units_l1': 11, 'n_units_l2': 91, 'n_units_l3': 1482, 'lr': 0.00035850095859304976, 'n_batchsize': 154}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:22:12,737] Trial 418 finished with value: 1195.60986328125 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 1370, 'n_units_l1': 13, 'n_units_l2': 117, 'n_units_l3': 1230, 'lr': 0.0002480135682569071, 'n_batchsize': 173}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:22:17,489] Trial 419 finished with value: 1564.275634765625 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1049, 'n_units_l1': 25, 'n_units_l2': 100, 'n_units_l3': 24, 'lr': 0.000463527098576277, 'n_batchsize': 160}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:22:23,559] Trial 420 finished with value: 1341.1021728515625 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 874, 'n_units_l1': 1992, 'n_units_l2': 86, 'n_units_l3': 1641, 'lr': 0.00017394600242925658, 'n_batchsize': 142}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:22:28,538] Trial 421 finished with value: 1194.5860595703125 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 1158, 'n_units_l1': 10, 'n_units_l2': 130, 'n_units_l3': 1338, 'lr': 0.0003206352322376976, 'n_batchsize': 166}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:22:34,485] Trial 422 finished with value: 1247.2236328125 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1242, 'n_units_l1': 15, 'n_units_l2': 61, 'n_units_l3': 1133, 'lr': 0.0004057669192661053, 'n_batchsize': 152}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:22:40,700] Trial 423 finished with value: 1216.62548828125 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 954, 'n_units_l1': 20, 'n_units_l2': 151, 'n_units_l3': 1490, 'lr': 0.00027266181425223535, 'n_batchsize': 134}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:22:46,314] Trial 424 finished with value: 1293.724609375 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 476, 'n_units_l1': 12, 'n_units_l2': 78, 'n_units_l3': 1668, 'lr': 0.0005936643399208672, 'n_batchsize': 158}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:22:51,725] Trial 425 finished with value: 1180.603759765625 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 1433, 'n_units_l1': 17, 'n_units_l2': 100, 'n_units_l3': 1351, 'lr': 0.00021731298281293883, 'n_batchsize': 146}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:22:56,980] Trial 426 finished with value: 1294.403564453125 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 1438, 'n_units_l1': 18, 'n_units_l2': 98, 'n_units_l3': 1812, 'lr': 0.00015831645516295516, 'n_batchsize': 179}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:23:02,084] Trial 427 finished with value: 1214.5218505859375 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 1557, 'n_units_l1': 17, 'n_units_l2': 109, 'n_units_l3': 1407, 'lr': 0.00021078475472381957, 'n_batchsize': 149}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:23:06,931] Trial 428 finished with value: 1221.1063232421875 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 1401, 'n_units_l1': 22, 'n_units_l2': 86, 'n_units_l3': 1996, 'lr': 0.00019979457438756062, 'n_batchsize': 167}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:23:12,336] Trial 429 finished with value: 1259.8033447265625 and parameters: {'n_layers': 5, 'n_epochs': 9, 'n_units_l0': 1295, 'n_units_l1': 16, 'n_units_l2': 102, 'n_units_l3': 1293, 'n_units_l4': 263, 'lr': 0.00023280421459282311, 'n_batchsize': 158}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:23:17,306] Trial 430 finished with value: 1416.86669921875 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 1621, 'n_units_l1': 20, 'n_units_l2': 68, 'n_units_l3': 1570, 'lr': 0.00012975776625543073, 'n_batchsize': 172}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:23:22,762] Trial 431 finished with value: 1245.057373046875 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 1063, 'n_units_l1': 15, 'n_units_l2': 89, 'n_units_l3': 1371, 'lr': 0.0002582907820518503, 'n_batchsize': 146}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:23:27,622] Trial 432 finished with value: 1203.2987060546875 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 1336, 'n_units_l1': 18, 'n_units_l2': 114, 'n_units_l3': 1755, 'lr': 0.0001840768345477765, 'n_batchsize': 163}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:23:30,668] Trial 433 finished with value: 1282.6083984375 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 884, 'n_units_l1': 14, 'n_units_l2': 96, 'n_units_l3': 1573, 'lr': 0.0002928037056709217, 'n_batchsize': 312}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:23:35,357] Trial 434 finished with value: 1355.7261962890625 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 298, 'n_units_l1': 16, 'n_units_l2': 77, 'n_units_l3': 1222, 'lr': 0.0002269980169295878, 'n_batchsize': 195}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:23:40,601] Trial 435 finished with value: 1257.402099609375 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 714, 'n_units_l1': 13, 'n_units_l2': 23, 'n_units_l3': 1100, 'lr': 0.0003431216166253822, 'n_batchsize': 155}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:23:46,395] Trial 436 finished with value: 1236.9608154296875 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1210, 'n_units_l1': 14, 'n_units_l2': 115, 'n_units_l3': 1424, 'lr': 0.0001545554965470989, 'n_batchsize': 152}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:23:52,589] Trial 437 finished with value: 1208.3458251953125 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1060, 'n_units_l1': 17, 'n_units_l2': 82, 'n_units_l3': 1789, 'lr': 0.00026577622990317133, 'n_batchsize': 143}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:23:56,231] Trial 438 finished with value: 1216.113525390625 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1509, 'n_units_l1': 19, 'n_units_l2': 99, 'n_units_l3': 1506, 'lr': 0.0003154375042168102, 'n_batchsize': 271}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:24:01,019] Trial 439 finished with value: 1324.0721435546875 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 964, 'n_units_l1': 12, 'n_units_l2': 121, 'n_units_l3': 1347, 'lr': 0.00018856271035450875, 'n_batchsize': 165}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:24:06,073] Trial 440 finished with value: 1243.5775146484375 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1201, 'n_units_l1': 15, 'n_units_l2': 107, 'n_units_l3': 1647, 'lr': 0.00022791775464136126, 'n_batchsize': 181}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:24:11,696] Trial 441 finished with value: 1223.0428466796875 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 811, 'n_units_l1': 13, 'n_units_l2': 88, 'n_units_l3': 1001, 'lr': 0.0003626796012728237, 'n_batchsize': 158}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:24:17,544] Trial 442 finished with value: 1497.123291015625 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1392, 'n_units_l1': 11, 'n_units_l2': 70, 'n_units_l3': 108, 'lr': 0.0002785322830462256, 'n_batchsize': 172}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:24:22,910] Trial 443 finished with value: 1214.44970703125 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 1076, 'n_units_l1': 16, 'n_units_l2': 128, 'n_units_l3': 1237, 'lr': 0.00022856171614838163, 'n_batchsize': 149}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:24:27,700] Trial 444 finished with value: 1188.47509765625 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1242, 'n_units_l1': 22, 'n_units_l2': 95, 'n_units_l3': 1499, 'lr': 0.00032608324018131035, 'n_batchsize': 189}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:24:33,926] Trial 445 finished with value: 1221.32080078125 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 970, 'n_units_l1': 14, 'n_units_l2': 109, 'n_units_l3': 1841, 'lr': 0.0003958401785083146, 'n_batchsize': 143}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:24:39,329] Trial 446 finished with value: 1213.075439453125 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1539, 'n_units_l1': 12, 'n_units_l2': 82, 'n_units_l3': 1151, 'lr': 0.0002776258838809934, 'n_batchsize': 163}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:24:44,543] Trial 447 finished with value: 1594.83251953125 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 14, 'n_units_l1': 18, 'n_units_l2': 134, 'n_units_l3': 1326, 'lr': 0.00020429112628910436, 'n_batchsize': 153}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:24:49,753] Trial 448 finished with value: 1220.5579833984375 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1121, 'n_units_l1': 13, 'n_units_l2': 103, 'n_units_l3': 1611, 'lr': 0.00016930379661397776, 'n_batchsize': 171}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:24:55,613] Trial 449 finished with value: 1185.919677734375 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1322, 'n_units_l1': 11, 'n_units_l2': 91, 'n_units_l3': 1438, 'lr': 0.00024794608936466224, 'n_batchsize': 158}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:25:00,819] Trial 450 finished with value: 1201.35986328125 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1604, 'n_units_l1': 11, 'n_units_l2': 119, 'n_units_l3': 1248, 'lr': 0.00024178132016465043, 'n_batchsize': 176}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:25:05,508] Trial 451 finished with value: 1427.6177978515625 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1371, 'n_units_l1': 10, 'n_units_l2': 93, 'n_units_l3': 218, 'lr': 0.00018502888717069826, 'n_batchsize': 160}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:25:10,873] Trial 452 finished with value: 1313.1939697265625 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1301, 'n_units_l1': 11, 'n_units_l2': 112, 'n_units_l3': 1989, 'lr': 0.00014309747060955725, 'n_batchsize': 165}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:25:13,518] Trial 453 finished with value: 1514.945068359375 and parameters: {'n_layers': 4, 'n_epochs': 4, 'n_units_l0': 1489, 'n_units_l1': 12, 'n_units_l2': 75, 'n_units_l3': 1052, 'lr': 0.0002670810693122738, 'n_batchsize': 155}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:25:18,673] Trial 454 finished with value: 1292.6776123046875 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 615, 'n_units_l1': 10, 'n_units_l2': 182, 'n_units_l3': 1421, 'lr': 0.0002256987178950486, 'n_batchsize': 169}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:25:23,619] Trial 455 finished with value: 1182.5833740234375 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1266, 'n_units_l1': 24, 'n_units_l2': 100, 'n_units_l3': 1637, 'lr': 0.000306353005042397, 'n_batchsize': 181}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:25:28,369] Trial 456 finished with value: 1202.1759033203125 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1401, 'n_units_l1': 25, 'n_units_l2': 104, 'n_units_l3': 1748, 'lr': 0.00031907832115422283, 'n_batchsize': 182}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:25:33,612] Trial 457 finished with value: 1207.07470703125 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1625, 'n_units_l1': 23, 'n_units_l2': 140, 'n_units_l3': 1666, 'lr': 0.00031649473440570667, 'n_batchsize': 195}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:25:36,064] Trial 458 finished with value: 1415.9056396484375 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1263, 'n_units_l1': 20, 'n_units_l2': 125, 'n_units_l3': 372, 'lr': 0.00037248265190494717, 'n_batchsize': 472}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:25:40,955] Trial 459 finished with value: 1202.248046875 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1298, 'n_units_l1': 20, 'n_units_l2': 96, 'n_units_l3': 1556, 'lr': 0.0002658809164737854, 'n_batchsize': 188}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:25:46,109] Trial 460 finished with value: 1426.5272216796875 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1460, 'n_units_l1': 11, 'n_units_l2': 114, 'n_units_l3': 1770, 'lr': 0.008562484318555576, 'n_batchsize': 181}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:25:51,276] Trial 461 finished with value: 1183.757568359375 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1219, 'n_units_l1': 22, 'n_units_l2': 100, 'n_units_l3': 1998, 'lr': 0.00030526352171643894, 'n_batchsize': 173}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:25:54,473] Trial 462 finished with value: 1215.0343017578125 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1167, 'n_units_l1': 25, 'n_units_l2': 124, 'n_units_l3': 1992, 'lr': 0.00039483491782888445, 'n_batchsize': 330}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:25:59,677] Trial 463 finished with value: 1198.447021484375 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1193, 'n_units_l1': 26, 'n_units_l2': 105, 'n_units_l3': 1844, 'lr': 0.0003217155771357059, 'n_batchsize': 178}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:26:04,569] Trial 464 finished with value: 1223.119140625 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1080, 'n_units_l1': 22, 'n_units_l2': 159, 'n_units_l3': 1810, 'lr': 0.0003005535211630713, 'n_batchsize': 187}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:26:09,758] Trial 465 finished with value: 1207.6513671875 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1229, 'n_units_l1': 21, 'n_units_l2': 102, 'n_units_l3': 1621, 'lr': 0.000376205032473419, 'n_batchsize': 175}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:26:14,392] Trial 466 finished with value: 1301.9757080078125 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1029, 'n_units_l1': 24, 'n_units_l2': 130, 'n_units_l3': 1651, 'lr': 0.000447921780537041, 'n_batchsize': 199}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:26:19,890] Trial 467 finished with value: 1259.82568359375 and parameters: {'n_layers': 5, 'n_epochs': 10, 'n_units_l0': 1382, 'n_units_l1': 19, 'n_units_l2': 114, 'n_units_l3': 1989, 'n_units_l4': 157, 'lr': 0.00028965134868144324, 'n_batchsize': 173}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:26:24,784] Trial 468 finished with value: 1288.148681640625 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 529, 'n_units_l1': 22, 'n_units_l2': 144, 'n_units_l3': 1574, 'lr': 0.000349311277874446, 'n_batchsize': 182}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:26:30,023] Trial 469 finished with value: 1201.3580322265625 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1155, 'n_units_l1': 17, 'n_units_l2': 89, 'n_units_l3': 1791, 'lr': 0.00028170309778662934, 'n_batchsize': 168}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:26:35,388] Trial 470 finished with value: 1417.6427001953125 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1647, 'n_units_l1': 20, 'n_units_l2': 97, 'n_units_l3': 1385, 'lr': 0.005300995070734662, 'n_batchsize': 166}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:26:39,961] Trial 471 finished with value: 1337.60400390625 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 150, 'n_units_l1': 27, 'n_units_l2': 109, 'n_units_l3': 1180, 'lr': 0.0004177482831027644, 'n_batchsize': 191}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:26:45,631] Trial 472 finished with value: 1271.1146240234375 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 978, 'n_units_l1': 17, 'n_units_l2': 121, 'n_units_l3': 1461, 'lr': 0.0003552684026660199, 'n_batchsize': 175}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:26:50,740] Trial 473 finished with value: 2050.779052734375 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 883, 'n_units_l1': 19, 'n_units_l2': 216, 'n_units_l3': 16, 'lr': 0.00025103525659956173, 'n_batchsize': 164}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:26:55,938] Trial 474 finished with value: 1308.069580078125 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1114, 'n_units_l1': 263, 'n_units_l2': 82, 'n_units_l3': 1659, 'lr': 0.0005200957389674792, 'n_batchsize': 172}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:27:00,495] Trial 475 finished with value: 1193.58056640625 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 1443, 'n_units_l1': 24, 'n_units_l2': 98, 'n_units_l3': 1293, 'lr': 0.00020228962099032242, 'n_batchsize': 160}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:27:05,587] Trial 476 finished with value: 1173.48876953125 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1280, 'n_units_l1': 13, 'n_units_l2': 138, 'n_units_l3': 921, 'lr': 0.00031960836407051953, 'n_batchsize': 181}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:27:10,512] Trial 477 finished with value: 1221.1656494140625 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1288, 'n_units_l1': 15, 'n_units_l2': 153, 'n_units_l3': 898, 'lr': 0.00023873961326695422, 'n_batchsize': 181}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:27:15,451] Trial 478 finished with value: 1197.0482177734375 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1525, 'n_units_l1': 13, 'n_units_l2': 173, 'n_units_l3': 1027, 'lr': 0.0002923700224987424, 'n_batchsize': 185}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:27:20,091] Trial 479 finished with value: 1195.54541015625 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 1263, 'n_units_l1': 22, 'n_units_l2': 135, 'n_units_l3': 1990, 'lr': 0.0003176046378396725, 'n_batchsize': 182}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:27:24,915] Trial 480 finished with value: 1211.2197265625 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1396, 'n_units_l1': 17, 'n_units_l2': 138, 'n_units_l3': 1075, 'lr': 0.00020383741141334928, 'n_batchsize': 187}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:27:29,838] Trial 481 finished with value: 1164.1842041015625 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1201, 'n_units_l1': 14, 'n_units_l2': 124, 'n_units_l3': 1506, 'lr': 0.00025384908686781443, 'n_batchsize': 177}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:27:34,532] Trial 482 finished with value: 1202.614501953125 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1181, 'n_units_l1': 15, 'n_units_l2': 128, 'n_units_l3': 1728, 'lr': 0.0002364598811650424, 'n_batchsize': 197}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:27:38,154] Trial 483 finished with value: 1268.292724609375 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 1591, 'n_units_l1': 14, 'n_units_l2': 147, 'n_units_l3': 1514, 'lr': 0.00016384461661780026, 'n_batchsize': 200}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:27:43,358] Trial 484 finished with value: 1215.2403564453125 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1314, 'n_units_l1': 18, 'n_units_l2': 117, 'n_units_l3': 1181, 'lr': 0.00018304887836236624, 'n_batchsize': 176}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:27:48,362] Trial 485 finished with value: 1229.53271484375 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1178, 'n_units_l1': 17, 'n_units_l2': 126, 'n_units_l3': 1822, 'lr': 0.00026072255547901583, 'n_batchsize': 177}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:27:53,144] Trial 486 finished with value: 1291.22509765625 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1503, 'n_units_l1': 408, 'n_units_l2': 161, 'n_units_l3': 1379, 'lr': 0.000214082675145378, 'n_batchsize': 190}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:27:57,820] Trial 487 finished with value: 1260.0655517578125 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 1310, 'n_units_l1': 16, 'n_units_l2': 120, 'n_units_l3': 918, 'lr': 0.00027316736999009693, 'n_batchsize': 171}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:28:03,420] Trial 488 finished with value: 1196.3343505859375 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1102, 'n_units_l1': 13, 'n_units_l2': 110, 'n_units_l3': 1589, 'lr': 0.00023134608968844678, 'n_batchsize': 182}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:28:08,324] Trial 489 finished with value: 1186.1668701171875 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1226, 'n_units_l1': 20, 'n_units_l2': 138, 'n_units_l3': 1150, 'lr': 0.00041087275990010015, 'n_batchsize': 192}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:28:13,787] Trial 490 finished with value: 1202.3751220703125 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1071, 'n_units_l1': 14, 'n_units_l2': 106, 'n_units_l3': 1304, 'lr': 0.00030102277112601067, 'n_batchsize': 168}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:28:18,485] Trial 491 finished with value: 1282.67138671875 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 1420, 'n_units_l1': 15, 'n_units_l2': 189, 'n_units_l3': 1807, 'lr': 0.000121675292817143, 'n_batchsize': 178}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:28:23,828] Trial 492 finished with value: 2341.55908203125 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1626, 'n_units_l1': 13, 'n_units_l2': 154, 'n_units_l3': 1522, 'lr': 1.223239609149531e-05, 'n_batchsize': 169}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:28:29,165] Trial 493 finished with value: 1395.1917724609375 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1184, 'n_units_l1': 18, 'n_units_l2': 116, 'n_units_l3': 1628, 'lr': 0.0031201385232004324, 'n_batchsize': 173}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:28:34,080] Trial 494 finished with value: 1253.423583984375 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1367, 'n_units_l1': 16, 'n_units_l2': 130, 'n_units_l3': 1416, 'lr': 0.00018050250576564557, 'n_batchsize': 186}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:28:39,147] Trial 495 finished with value: 1188.526611328125 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 1080, 'n_units_l1': 12, 'n_units_l2': 100, 'n_units_l3': 1256, 'lr': 0.00026939095475501744, 'n_batchsize': 164}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:28:44,644] Trial 496 finished with value: 1192.9923095703125 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1257, 'n_units_l1': 22, 'n_units_l2': 112, 'n_units_l3': 1981, 'lr': 0.0003562831616256159, 'n_batchsize': 161}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:28:49,819] Trial 497 finished with value: 1209.3924560546875 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1470, 'n_units_l1': 14, 'n_units_l2': 135, 'n_units_l3': 994, 'lr': 0.000212750275251584, 'n_batchsize': 174}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:28:55,664] Trial 498 finished with value: 1335.4161376953125 and parameters: {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1028, 'n_units_l1': 20, 'n_units_l2': 103, 'n_units_l3': 176, 'lr': 0.0004574953211061192, 'n_batchsize': 153}. Best is trial 321 with value: 1159.6004638671875.
[I 2022-02-03 18:28:59,319] Trial 499 finished with value: 1199.5865478515625 and parameters: {'n_layers': 4, 'n_epochs': 9, 'n_units_l0': 1173, 'n_units_l1': 153, 'n_units_l2': 118, 'n_units_l3': 1698, 'lr': 0.000324189824749899, 'n_batchsize': 241}. Best is trial 321 with value: 1159.6004638671875.

The best trial can then be assessed.

In [38]:
print("Best params: ", study.best_params)
print("Best value: ", study.best_value)
print("Best Trial: ", study.best_trial)
Best params:  {'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1167, 'n_units_l1': 14, 'n_units_l2': 111, 'n_units_l3': 1502, 'lr': 0.00034296931053536195, 'n_batchsize': 161}
Best value:  1159.6004638671875
Best Trial:  FrozenTrial(number=321, values=[1159.6004638671875], datetime_start=datetime.datetime(2022, 2, 3, 18, 13, 40, 855459), datetime_complete=datetime.datetime(2022, 2, 3, 18, 13, 46, 323793), params={'n_layers': 4, 'n_epochs': 10, 'n_units_l0': 1167, 'n_units_l1': 14, 'n_units_l2': 111, 'n_units_l3': 1502, 'lr': 0.00034296931053536195, 'n_batchsize': 161}, distributions={'n_layers': IntUniformDistribution(high=5, low=1, step=1), 'n_epochs': IntUniformDistribution(high=10, low=1, step=1), 'n_units_l0': IntLogUniformDistribution(high=2000, low=10, step=1), 'n_units_l1': IntLogUniformDistribution(high=2000, low=10, step=1), 'n_units_l2': IntLogUniformDistribution(high=2000, low=10, step=1), 'n_units_l3': IntLogUniformDistribution(high=2000, low=10, step=1), 'lr': LogUniformDistribution(high=0.1, low=1e-05), 'n_batchsize': IntUniformDistribution(high=512, low=128, step=1)}, user_attrs={}, system_attrs={}, intermediate_values={}, trial_id=321, state=TrialState.COMPLETE, value=None)

Here comes the 3rd point about Optuna which I like - very verbose. Optuna has amazing visualizations!

In [39]:
import plotly
In [40]:
from optuna.visualization import plot_optimization_history
plot_optimization_history(study)