Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

onedb fsck not count leases from reserverd networks #1710

Closed
4 of 26 tasks
kvaps opened this issue Feb 3, 2018 · 10 comments
Closed
4 of 26 tasks

onedb fsck not count leases from reserverd networks #1710

kvaps opened this issue Feb 3, 2018 · 10 comments
Assignees
@al3xhh
Labels
Category: CLI Community Priority: Normal Status: Accepted Type: Bug
Milestone
Release 5.10

Comments

@kvaps
Copy link
Contributor

kvaps commented Feb 3, 2018 •
edited by juanmont
Loading

Bug Report

Version of OpenNebula

  • 5.2.2
  • 5.4.0
  • 5.4.1
  • 5.4.2
  • 5.4.3
  • 5.4.4
  • 5.4.5
  • 5.4.6
  • 5.4.7
  • Development build

Component

  • Authorization (LDAP, x509 certs...)
  • Command Line Interface (CLI)
  • Contextualization
  • Documentation
  • Federation and HA
  • Host, Clusters and Monitorization
  • KVM
  • Networking
  • Orchestration (OpenNebula Flow)
  • Packages
  • Scheduler
  • Storage & Images
  • Sunstone
  • Upgrades
  • User, Groups, VDCs and ACL
  • vCenter

Description

I have virtual network with 23, and network reservation for each user from it.

Expected Behavior

onedb fsck checks network leases in reserved network too.

Actual Behavior

onedb fsck break network quotas set it to "0":

User 350 quotas: VNet 23        LEASES_USED has 1       is      0
User 351 quotas: VNet 23        LEASES_USED has 1       is      0
User 352 quotas: VNet 23        LEASES_USED has 1       is      0
User 353 quotas: VNet 23        LEASES_USED has 1       is      0
User 355 quotas: VNet 23        LEASES_USED has 1       is      0
User 356 quotas: VNet 23        LEASES_USED has 1       is      0
Group 150 quotas: VNet 23       LEASES_USED has 1       is      0
Group 151 quotas: VNet 23       LEASES_USED has 1       is      0
Group 152 quotas: VNet 23       LEASES_USED has 1       is      0
Group 153 quotas: VNet 23       LEASES_USED has 1       is      0
Group 155 quotas: VNet 23       LEASES_USED has 1       is      0
Group 156 quotas: VNet 23       LEASES_USED has 1       is      0

How to reproduce

  • Create network reservation
  • Run onedb fsck
@kvaps
Copy link
Contributor Author

kvaps commented Jan 17, 2019 •
edited
Loading

Issue is still there on 5.6.1

@kvaps
Copy link
Contributor Author

kvaps commented Jan 17, 2019

Patch for disable this check:

--- a/usr/lib/one/ruby/onedb/fsck/quotas.rb
+++ b/usr/lib/one/ruby/onedb/fsck/quotas.rb
@@ -273,6 +273,8 @@
             net_quota = doc.root.add_child(doc.create_element("NETWORK_QUOTA"))
         end
 
+        # Disabled due https://github.com/OpenNebula/one/issues/1710
+=begin
         net_quota.xpath("NETWORK").each { |net_elem|
             vnet_id = net_elem.at_xpath("ID").text
 
@@ -297,6 +299,7 @@
             new_elem.add_child(doc.create_element("LEASES")).content = "-1"
             new_elem.add_child(doc.create_element("LEASES_USED")).content = leases_used.to_s
         }
+=end
 
         # Image quotas

@al3xhh al3xhh self-assigned this May 27, 2019
@al3xhh al3xhh added this to the Release 5.10 milestone May 27, 2019
al3xhh pushed a commit to al3xhh/one that referenced this issue May 31, 2019
B OpenNebula#1710: network fsck takes reserved networks
097e270
al3xhh pushed a commit to al3xhh/one that referenced this issue Jun 3, 2019
B OpenNebula#1710: network fsck takes reserved networks
5601a80
@al3xhh
Copy link
Contributor

al3xhh commented Jun 3, 2019

PRs to merge in 5.8 and master:

code: #3398
docs: OpenNebula/docs#641

atodorov-storpool added a commit to atodorov-storpool/one that referenced this issue Jun 4, 2019
@atodorov-storpool
B OpenNebula#1710: do not stringify an array ;)
c5601ac 
It is possible to have more than one AR in an AR_POOL
So we need to sum the sizes of each AR
rsmontero pushed a commit that referenced this issue Jun 4, 2019
@rsmontero
B #1710: network fsck takes reserved networks (#3398)
b16ea97
rsmontero pushed a commit that referenced this issue Jun 4, 2019
@rsmontero
B #1710: network fsck takes reserved networks (#3398)
6c041cf 
(cherry picked from commit b16ea97)
rsmontero pushed a commit to OpenNebula/docs that referenced this issue Jun 4, 2019
B OpenNebula/one#1710: add bug to resolved issues (#641)
977557f
@atodorov-storpool
Copy link
Contributor

Just dropping a note that without atodorov-storpool@c5601ac
this patch do not count the sizes of the ARs in an AR_POOL.
It concatenate them instead!
For example if you have 2 ARs with 253 IPs in an AR_POOL, this will update with 253253 instead of 253+253=506

al3xhh pushed a commit to al3xhh/one that referenced this issue Jun 5, 2019
B OpenNebula#1710: fix bug when parent_id is empty
e378cb3
@al3xhh
Copy link
Contributor

al3xhh commented Jun 5, 2019

Thanks @atodorov-storpool I have added it and I have also fixed another bug.

rsmontero pushed a commit that referenced this issue Jun 5, 2019
@rsmontero
B #1710: fix bug when parent_id is empty (#3405)
7dc16bf
rsmontero pushed a commit that referenced this issue Jun 5, 2019
@rsmontero
B #1710: fix bug when parent_id is empty (#3405)
29ff8d8 
(cherry picked from commit 7dc16bf)
tinova added a commit to OpenNebula/docs that referenced this issue Jun 19, 2019
@tinova
Squashed commit of the following:
bc7c142 
commit 9e339df
Author: Tino Vazquez <[email protected]>
Date:   Mon Jun 17 16:58:49 2019 +0200

    M #-: RNs for 5.8.3

commit 1c5c4da
Author: Tino Vazquez <[email protected]>
Date:   Mon Jun 17 13:05:46 2019 +0200

    M #-: Reduce number of upgrade guides

    Two new categories, upgrading from 4.x.x and from 3.x.x

commit 5834470
Author: Christian González <[email protected]>
Date:   Mon Jun 17 09:59:09 2019 +0200

    F #3393: Add references to DB maintenance section (#665)

    (cherry picked from commit 8ba5fde)

commit 807f50a
Author: Jorge Lobo <[email protected]>
Date:   Fri Jun 14 18:13:05 2019 +0200

    B #3430: fix bug lost icons VNC in federation (#663)

    Signed-off-by: Jorge Lobo <[email protected]>

commit 2ac9001
Author: Christian González <[email protected]>
Date:   Fri Jun 14 15:55:10 2019 +0200

    F #3393: Add information about DB maintenance (#655)

commit d655ea7
Author: Christian González <[email protected]>
Date:   Fri Jun 14 15:42:37 2019 +0200

    F #1324: Option to not store monitoring data to the DB (#656)

commit d7b5cea
Author: Angel Luis Moya Gonzalez <[email protected]>
Date:   Fri Jun 14 13:37:10 2019 +0200

    B #3225: Add IDE management explanation

commit ea703be
Author: Tino Vázquez <[email protected]>
Date:   Fri Jun 14 13:32:57 2019 +0200

    B #3430: Add resolved issues reference

commit 67f06f9
Author: Angel Luis Moya Gonzalez <[email protected]>
Date:   Fri Jun 14 13:05:46 2019 +0200

    Update resolved_issues_583.rst (#661)

commit d961a6c
Author: Ruben S. Montero <[email protected]>
Date:   Wed Jun 12 17:49:41 2019 +0200

    B OpenNebula/one#3393: Reduce DB disk space

commit 9258381
Author: Daniel Clavijo Coca <[email protected]>
Date:   Tue Jun 11 04:49:41 2019 -0500

    F #3406: Update resolved issues (#653)

commit 08781b9
Author: Daniel Clavijo Coca <[email protected]>
Date:   Tue Jun 11 04:47:05 2019 -0500

    B #3390: Update resolved issues (#654)

commit aacb7e4
Author: Alejandro Huertas Herrero <[email protected]>
Date:   Mon Jun 10 12:25:55 2019 +0200

    F OpenNebula/one#3378: add link to the new onezone.yaml (#652)

commit 1c8d842
Author: Alejandro Huertas Herrero <[email protected]>
Date:   Mon Jun 10 12:00:59 2019 +0200

    F OpenNebula/one#3378: add feature to resolved issues (#651)

    * F OpenNebula/one#3378: add feature to resolved issues

    * Update resolved_issues_583.rst

commit bf7473a
Author: Ruben S. Montero <[email protected]>
Date:   Fri Jun 7 16:59:21 2019 +0200

    Update resolved_issues_583.rst

commit 977557f
Author: Alejandro Huertas Herrero <[email protected]>
Date:   Tue Jun 4 10:09:18 2019 +0200

    B OpenNebula/one#1710: add bug to resolved issues (#641)

commit 015f924
Author: Jorge Lobo <[email protected]>
Date:   Mon Jun 3 17:23:33 2019 +0200

    B # OpenNebula/one#3057: add resolved_issue (#640)

    Signed-off-by: Jorge Lobo <[email protected]>

commit 89ac977
Author: Tino Vazquez <[email protected]>
Date:   Mon Jun 3 13:44:57 2019 +0200

    F #3366: Add feature to release notes

    Now a group base can be different from user base

commit cebc55c
Author: Steven Armstrong <[email protected]>
Date:   Mon Jun 3 13:40:52 2019 +0200

    M #: document new :group_base ldap config (#638)

    implemented in OpenNebula/one#3379

    Signed-off-by: Steven Armstrong <[email protected]>
    (cherry picked from commit 33d8a0d)

commit 9c7225b
Author: Alejandro Huertas Herrero <[email protected]>
Date:   Mon Jun 3 13:03:52 2019 +0200

    B OpenNebula/one#2047: add bug to resolved issues (#642)

commit b4daae2
Author: Jorge Lobo <[email protected]>
Date:   Thu May 30 18:24:35 2019 +0200

    B # OpenNebula/one#3368: add bug (#637)

    Signed-off-by: Jorge Lobo <[email protected]>

commit e4d232d
Author: Daniel Clavijo Coca <[email protected]>
Date:   Thu May 30 03:35:24 2019 -0500

    M #: doc required one restart (#636)

commit 7ee5175
Author: Alejandro Huertas Herrero <[email protected]>
Date:   Tue May 28 17:26:52 2019 +0200

    B OpenNebula/one#3269: add bug to resolved issues (#635)

commit d10efbb
Author: Ruben S. Montero <[email protected]>
Date:   Mon May 27 13:05:45 2019 +0200

    Update resolved_issues_583.rst

commit 808b6bd
Author: Tino Vazquez <[email protected]>
Date:   Fri May 24 12:44:55 2019 +0200

    F #964: Remove FILES in vCenter context limitation

commit be2c062
Author: Tino Vazquez <[email protected]>
Date:   Fri May 24 12:30:49 2019 +0200

    F #964: Add note in RNs

commit 8fcdf2e
Author: Alejandro Huertas Herrero <[email protected]>
Date:   Thu May 23 11:44:48 2019 +0200

    B OpenNebula/one#3357: add bug to resolved issues (#633)

commit 6f86bb4
Author: Tino Vazquez <[email protected]>
Date:   Tue May 21 11:16:14 2019 +0200

    B #3335: Add note on Sunstone vector attributes

    (cherry picked from commit eb194c8)

commit 52359cc
Author: Tino Vazquez <[email protected]>
Date:   Tue May 21 11:12:50 2019 +0200

    B #3335: Add user template Sunstone confs

commit 840edcb
Author: sergiojvg <[email protected]>
Date:   Mon May 20 11:43:25 2019 -0500

    F #3337: Resolved issues (#630)

commit bca1a32
Author: Daniel Clavijo Coca <[email protected]>
Date:   Mon May 20 04:53:00 2019 -0500

    F OpenNebula#3222: updated resolved issues (#626)

commit 5bd33c4
Author: Tino Vazquez <[email protected]>
Date:   Fri May 17 18:29:00 2019 +0200

    B #3331: Support token behind proxy

commit 501a028
Author: jmdelafe <[email protected]>
Date:   Fri May 17 11:51:17 2019 -0400

    B OpenNebula/one#3280 and B OpenNebula/one#3281 fixed (#629)

    * Update resolved_issues_583.rst

    * Update resolved_issues_583.rst

    * Update resolved_issues_583.rst

commit 1d1b563
Author: Vlastimil Holer <[email protected]>
Date:   Tue May 14 17:21:44 2019 +0200

    M #-: Bump version to 5.8.3, scaffolding (#625)

commit 8dfe0c1
Author: Tino Vazquez <[email protected]>
Date:   Tue May 14 17:03:18 2019 +0200

    M #-: Fix hotfix box for 5.8.2

    (cherry picked from commit 51d5a82)
@kvaps
Copy link
Contributor Author

kvaps commented Aug 23, 2019

BTW, this patch is not working for me, still see the same:

User 2774 quotas: VNet 23	LEASES_USED has 1 	is	0
User 2774 quotas: VNet 2	LEASES_USED has 0 	is	1
User 2775 quotas: VNet 23	LEASES_USED has 1 	is	0
User 2775 quotas: VNet 5	LEASES_USED has 0 	is	1
User 2776 quotas: VNet 23	LEASES_USED has 1 	is	0
User 2776 quotas: VNet 6	LEASES_USED has 0 	is	1
User 2777 quotas: VNet 23	LEASES_USED has 1 	is	0
User 2777 quotas: VNet 7	LEASES_USED has 0 	is	1
User 2778 quotas: VNet 23	LEASES_USED has 1 	is	0
User 2778 quotas: VNet 6	LEASES_USED has 0 	is	1
User 2779 quotas: VNet 23	LEASES_USED has 1 	is	0
User 2779 quotas: VNet 2	LEASES_USED has 0 	is	1

and for groups:

Group 2571 quotas: VNet 5	LEASES_USED has 0 	is	1
Group 2572 quotas: VNet 23	LEASES_USED has 1 	is	0
Group 2572 quotas: VNet 6	LEASES_USED has 0 	is	1
Group 2573 quotas: VNet 23	LEASES_USED has 1 	is	0
Group 2573 quotas: VNet 7	LEASES_USED has 0 	is	1
Group 2574 quotas: VNet 23	LEASES_USED has 1 	is	0
Group 2574 quotas: VNet 6	LEASES_USED has 0 	is	1
Group 2575 quotas: VNet 23	LEASES_USED has 1 	is	0
Group 2575 quotas: VNet 2	LEASES_USED has 0 	is	1

version compiled from 5.8.4, submitted patch persists in the code

@kvaps
Copy link
Contributor Author

kvaps commented Aug 23, 2019 •
edited
Loading

  • b16ea97 - This patch is working, but records them into empty </ID>
  • e378cb3 (from master) - This patch in not working at all

@al3xhh FYI

@al3xhh
Copy link
Contributor

al3xhh commented Sep 2, 2019

Hello @kvaps tanks for trying it!

Could you please elaborate more about this: but records them into empty </ID>, do you mean that the ID of the reserved network is removed?

About this e378cb3 could you please let me know what errors do you get? I have checked and for me it's working, it detects the reserved networks.

@kvaps
Copy link
Contributor Author

kvaps commented Mar 10, 2020 •
edited
Loading

@al3xhh eg, I have

Group 3133 quotas: VNet 23	LEASES_USED has 6 	is	0
Group 3133 quotas: VNet 111111	LEASES_USED has 0 	is	6

but in db:

(3133,'<QUOTAS><ID>3133</ID><DATASTORE_QUOTA></DATASTORE_QUOTA><NETWORK_QUOTA><NETWORK><ID><![CDATA[23]]></ID><LEASES><![CDATA[6]]></LEASES><LEASES_USED><![CDATA[6]]></LEASES_USED></NETWORK><NETWORK><ID><![CDATA[5432]]></ID><LEASES><![CDATA[-1]]></LEASES><LEASES_USED><![CDATA[5]]></LEASES_USED></NETWORK><NETWORK><ID><![CDATA[5433]]></ID><LEASES><![CDATA[-1]]></LEASES><LEASES_USED><![CDATA[1]]></LEASES_USED></NETWORK><NETWORK><ID><![CDATA[5434]]></ID><LEASES><![CDATA[-1]]></LEASES><LEASES_USED><![CDATA[1]]></LEASES_USED></NETWORK></NETWORK_QUOTA><VM_QUOTA><VM><CPU><![CDATA[0.34]]></CPU><CPU_USED><![CDATA[0.33]]></CPU_USED><MEMORY><![CDATA[2048]]></MEMORY><MEMORY_USED><![CDATA[2048]]></MEMORY_USED><RUNNING_CPU><![CDATA[-1]]></RUNNING_CPU><RUNNING_CPU_USED><![CDATA[0.33]]></RUNNING_CPU_USED><RUNNING_MEMORY><![CDATA[-1]]></RUNNING_MEMORY><RUNNING_MEMORY_USED><![CDATA[2048]]></RUNNING_MEMORY_USED><RUNNING_VMS><![CDATA[-1]]></RUNNING_VMS><RUNNING_VMS_USED><![CDATA[1]]></RUNNING_VMS_USED><SYSTEM_DISK_SIZE><![CDATA[30720]]></SYSTEM_DISK_SIZE><SYSTEM_DISK_SIZE_USED><![CDATA[30720]]></SYSTEM_DISK_SIZE_USED><VMS><![CDATA[1]]></VMS><VMS_USED><![CDATA[1]]></VMS_USED></VM></VM_QUOTA><IMAGE_QUOTA><IMAGE><ID><![CDATA[2054]]></ID><RVMS><![CDATA[-1]]></RVMS><RVMS_USED><![CDATA[1]]></RVMS_USED></IMAGE></IMAGE_QUOTA></QUOTAS>')

I have no any network with id 111111

The only value 111111 in my db is another record in group_quotas table, where 111111 in LEASES_USED:

(807,'<QUOTAS>\n  <ID>807</ID>\n  <DATASTORE_QUOTA/>\n  <NETWORK_QUOTA>\n    <NETWORK>\n      <ID><![CDATA[23]]></ID>\n      <LEASES><![CDATA[6]]></LEASES>\n      <LEASES_USED>6</LEASES_USED>\n    </NETWORK>\n    <NETWORK>\n      <ID><![CDATA[736]]></ID>\n      <LEASES><![CDATA[-1]]></LEASES>\n      <LEASES_USED><![CDATA[6]]></LEASES_USED>\n    </NETWORK>\n    <NETWORK>\n      <ID>313333</ID>\n      <LEASES>-1</LEASES>\n      <LEASES_USED>111111</LEASES_USED>\n    </NETWORK>\n  </NETWORK_QUOTA>\n  <VM_QUOTA>\n    <VM>\n      <CPU><![CDATA[6.01]]></CPU>\n      <CPU_USED><![CDATA[6]]></CPU_USED>\n      <MEMORY><![CDATA[16384]]></MEMORY>\n      <MEMORY_USED><![CDATA[16384]]></MEMORY_USED>\n      <RUNNING_CPU><![CDATA[-1]]></RUNNING_CPU>\n      <RUNNING_CPU_USED>6.00</RUNNING_CPU_USED>\n      <RUNNING_MEMORY><![CDATA[-1]]></RUNNING_MEMORY>\n      <RUNNING_MEMORY_USED>16384</RUNNING_MEMORY_USED>\n      <RUNNING_VMS><![CDATA[-1]]></RUNNING_VMS>\n      <RUNNING_VMS_USED>1</RUNNING_VMS_USED>\n      <SYSTEM_DISK_SIZE><![CDATA[491520]]></SYSTEM_DISK_SIZE>\n      <SYSTEM_DISK_SIZE_USED><![CDATA[491520]]></SYSTEM_DISK_SIZE_USED>\n      <VMS><![CDATA[1]]></VMS>\n      <VMS_USED><![CDATA[1]]></VMS_USED>\n    </VM>\n  </VM_QUOTA>\n  <IMAGE_QUOTA>\n    <IMAGE>\n      <ID><![CDATA[80]]></ID>\n      <RVMS><![CDATA[-1]]></RVMS>\n      <RVMS_USED><![CDATA[1]]></RVMS_USED>\n    </IMAGE>\n  </IMAGE_QUOTA>\n</QUOTAS>')

Group 3134 quotas: VNet 23	LEASES_USED has 1 	is	0
Group 3134 quotas: VNet 4	LEASES_USED has 0 	is	1

in db

(3134,'<QUOTAS><ID>3134</ID><DATASTORE_QUOTA></DATASTORE_QUOTA><NETWORK_QUOTA><NETWORK><ID><![CDATA[23]]></ID><LEASES><![CDATA[1]]></LEASES><LEASES_USED><![CDATA[1]]></LEASES_USED></NETWORK><NETWORK><ID><![CDATA[5436]]></ID><LEASES><![CDATA[-1]]></LEASES><LEASES_USED><![CDATA[1]]></LEASES_USED></NETWORK><NETWORK><ID><![CDATA[5437]]></ID><LEASES><![CDATA[-1]]></LEASES><LEASES_USED><![CDATA[1]]></LEASES_USED></NETWORK></NETWORK_QUOTA><VM_QUOTA><VM><CPU><![CDATA[0.34]]></CPU><CPU_USED><![CDATA[0.33]]></CPU_USED><MEMORY><![CDATA[512]]></MEMORY><MEMORY_USED><![CDATA[512]]></MEMORY_USED><RUNNING_CPU><![CDATA[-1]]></RUNNING_CPU><RUNNING_CPU_USED><![CDATA[-3.63]]></RUNNING_CPU_USED><RUNNING_MEMORY><![CDATA[-1]]></RUNNING_MEMORY><RUNNING_MEMORY_USED><![CDATA[-5632]]></RUNNING_MEMORY_USED><RUNNING_VMS><![CDATA[-1]]></RUNNING_VMS><RUNNING_VMS_USED><![CDATA[-11]]></RUNNING_VMS_USED><SYSTEM_DISK_SIZE><![CDATA[5120]]></SYSTEM_DISK_SIZE><SYSTEM_DISK_SIZE_USED><![CDATA[5120]]></SYSTEM_DISK_SIZE_USED><VMS><![CDATA[1]]></VMS><VMS_USED><![CDATA[1]]></VMS_USED></VM></VM_QUOTA><IMAGE_QUOTA><IMAGE><ID><![CDATA[1912]]></ID><RVMS><![CDATA[-1]]></RVMS><RVMS_USED><![CDATA[1]]></RVMS_USED></IMAGE></IMAGE_QUOTA></QUOTAS>')

I have no any vnet with id 4, I don't know where is it taking this network ids.

@kvaps
Copy link
Contributor Author

kvaps commented Mar 10, 2020

got it, I'll report it in new issue

@kvaps kvaps mentioned this issue Mar 10, 2020
Closed
7 tasks
rsmontero pushed a commit that referenced this issue Jan 24, 2022
M #~: Fix minor to header translates (#1710)
44e27be
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@al3xhh al3xhh

Labels
Category: CLI Community Priority: Normal Status: Accepted Type: Bug
Projects
None yet
Milestone
Release 5.10
Development

No branches or pull requests
5 participants
@rsmontero @kvaps @atodorov-storpool @juanmont @al3xhh