Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci and delta using tf2.3.0 #235

Merged
merged 15 commits into from
Nov 10, 2020
Merged

ci and delta using tf2.3.0 #235

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
7824852
ci and delta using tf2.3.0
zh794390558 Aug 27, 2020
3c16eff
fix dockerfile using tf2.3.0
zh794390558 Aug 27, 2020
44e8508
using tf2.3.0 docker image
zh794390558 Aug 28, 2020
a2237eb
update readme
zh794390558 Aug 28, 2020
0852dff
deprecated export_saved_model
zh794390558 Aug 28, 2020
1a90836
During the transition from TF 1 to TF 2 you can use `tf.compat.v1.da…
zh794390558 Aug 28, 2020
8649f97
fix bug for text_ops
Nov 9, 2020
34f4f08
update load_text_line_dataset
Nov 9, 2020
1a83479
Merge pull request #243 from yantingxu/docker
zh794390558 Nov 9, 2020
4b22851
fix doc
zh794390558 Nov 9, 2020
d9ab566
fix speech splice raise error
zh794390558 Nov 9, 2020
3094aeb
update for rouge api changes
Nov 9, 2020
c3646de
asr test metric used error
zh794390558 Nov 9, 2020
4f002df
fix sph2pipe install
zh794390558 Nov 10, 2020
e521d13
fix
zh794390558 Nov 10, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
During the transition from TF 1 to TF 2 you can use `tf.compat.v1.da… 
…ta.make_initializable_iterator(dataset)` to create a TF 1 graph mode style iterator for a dataset created through TF 2 APIs.
  • Loading branch information
@zh794390558
zh794390558 committed Aug 28, 2020
commit 1a90836897e63cc63c0a2c7c9c74af5b091de8cb
4 changes: 2 additions & 2 deletions delta/data/preprocess/base_preparer.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -144,7 +144,7 @@ def prepare_one_raw_data(self, one_path, one_path_after, mode,
if self.multi_output:
for i in range(self.output_num):
label_ds = label[i].batch(self.batch_size)
label_iterator = label_ds.make_initializable_iterator()
label_iterator = tf.data.make_initializable_iterator(label_ds)
label_after_arr = self.run_dataset(label_iterator, batch_num)
label_after_one = [
one_line.decode("utf-8") for one_line in label_after_arr
Expand All @@ -154,7 +154,7 @@ def prepare_one_raw_data(self, one_path, one_path_after, mode,
else:
label = label[0]
label_ds = label.batch(self.batch_size)
label_iterator = label_ds.make_initializable_iterator()
label_iterator = tf.data.make_initializable_iterator(label_ds)
label_after_arr = self.run_dataset(label_iterator, batch_num)
one_label_after = [
one_line.decode("utf-8") for one_line in label_after_arr
Expand Down
4 changes: 2 additions & 2 deletions delta/data/preprocess/text_ops_test.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -121,7 +121,7 @@ def test_process_one_label_dataset(self):
true_res = [0, 0, 0, 8]
label_ds = process_one_label_dataset(label_ds, self.config)

iterator = label_ds.make_initializable_iterator()
iterator = tf.data.make_initializable_iterator(label_ds)
label_res = iterator.get_next()

with tf.Session() as sess:
Expand All @@ -139,7 +139,7 @@ def test_process_multi_label_dataset(self):
label_ds = tf.data.TextLineDataset(label_filepath)
true_res = [[0, 8, 8], [0, 7, 8]]
label_ds = process_multi_label_dataset(label_ds, self.config)
iterator = label_ds.make_initializable_iterator()
iterator = tf.data.make_initializable_iterator(label_ds)
label_res = iterator.get_next()

with tf.Session() as sess:
Expand Down
2 changes: 1 addition & 1 deletion delta/data/preprocess/utils.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ def get_pre_process_text_ds_iter(

text_ds = text_ds.batch(batch_size)

iterator = text_ds.make_initializable_iterator()
iterator = tf.data.make_initializable_iterator(text_ds)

return iterator

Expand Down
2 changes: 1 addition & 1 deletion delta/data/task/text_cls_task.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -184,7 +184,7 @@ def dataset(self):

data_set = data_set.prefetch(self.num_prefetch_batch)

iterator = data_set.make_initializable_iterator()
iterator = tf.data.make_initializable_iterator(data_set)

# pylint: disable=unused-variable
if self.infer_without_label:
Expand Down
2 changes: 1 addition & 1 deletion delta/data/task/text_match_task.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -170,7 +170,7 @@ def dataset(self):

text_ds_left_right = text_ds_left_right.prefetch(self.num_prefetch_batch)

iterator = text_ds_left_right.make_initializable_iterator()
iterator = tf.data.make_initializable_iterator(text_ds_left_right)
# pylint: disable=unused-variable
if self.infer_without_label:
(input_x_left, input_x_right), (input_x_left_len,
Expand Down
2 changes: 1 addition & 1 deletion delta/data/task/text_nlu_joint_task.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -160,7 +160,7 @@ def dataset(self):

data_set = data_set.prefetch(self.num_prefetch_batch)

iterator = data_set.make_initializable_iterator()
iterator = tf.data.make_initializable_iterator(data_set)

if self.infer_without_label:
input_x, input_x_len = iterator.get_next()
Expand Down
2 changes: 1 addition & 1 deletion delta/data/task/text_seq2seq_task.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -224,7 +224,7 @@ def dataset(self):

data_set = data_set.prefetch(self.num_prefetch_batch)

iterator = data_set.make_initializable_iterator()
iterator = tf.data.make_initializable_iterator(data_set)

# pylint: disable=unused-variable
if self.infer_without_label:
Expand Down
2 changes: 1 addition & 1 deletion delta/data/task/text_seq_label_task.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -139,7 +139,7 @@ def dataset(self):

data_set = data_set.prefetch(self.num_prefetch_batch)

iterator = data_set.make_initializable_iterator()
iterator = tf.data.make_initializable_iterator(data_set)

if self.infer_without_label:
input_x, input_x_len = iterator.get_next()
Expand Down