Skip to content

Commit

Permalink
Rename project to FogTracker. Strip out Billing code and ActiveRecord
Browse files Browse the repository at this point in the history
  • Loading branch information
benton committed Jan 22, 2012
1 parent 4f25056 commit 8274dcc
Show file tree
Hide file tree
Showing 24 changed files with 150 additions and 413 deletions.
2 changes: 1 addition & 1 deletion Gemfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
source "http://rubygems.org"

# Specify your gem's dependencies in cloud_cost_tracker.gemspec
# Specify your gem's dependencies in fog_tracker.gemspec
gemspec
85 changes: 25 additions & 60 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,69 +1,40 @@
Cloud Cost Tracker
Fog Tracker
================
Generates BillingRecords (ActiveRecords) for each cloud computing resource
discovered by the [Fog gem](https://github.com/fog/fog)
Uses the Fog gem to track the state of cloud computing resources across multiple accounts, with multiple service providers.

*ALPHA VERSION - not yet fully functional*


----------------
What is it?
----------------
The Cloud Cost Tracker periodically polls one or more cloud computing accounts and determines the state of their associated cloud computing "resources": compute instances, disk volumes, stored objects, and so on. Each time the accounts are queried, an ActiveRecord object (a BillingRecord) is created or updated for each resource, containing the cost for that resource over the period since its previous BillingRecord.
The Fog Tracker uses the [Fog gem](https://github.com/fog/fog) to periodically poll one or more cloud computing accounts, and to determine the state of their associated cloud computing "Resources": compute instances, disk volumes, stored objects, and so on. The most recent state of all Resources is saved in memory (as Fog objects), and can be queried repeatedly in ways similar to Fog, but with no network overhead.


----------------
Why is it?
----------------
The Cloud Cost Tracker is intended to be a foundation library, on top of which more complex cloud billing / accounting applications can be built. Although an executable 'tracker' command-line program is included, the library is primarily intended for use from within Rails, or some other ActiveRecord context, that can generate reports based on the BillingRecords.
The Fog Tracker is intended to be a foundation library, on top of which more complex cloud dashboard or management applications can be built. It allows such applications to decouple their requests to cloud service providers from their access to the results of those requests.


----------------
Installation
Where is it? (Installation)
----------------
Install the Cloud Cost Tracker gem and your database adaptor of choice.
Install the Fog Tracker gem from RubyGems

gem install cloud_cost_tracker sqlite3
gem install fog_tracker


----------------
Usage [from within Ruby]
How is it [done]? (Usage)
----------------
1) Add the BillingRecords table into your database.
Just put `require 'CloudCostTracker/tasks'` in your Rakefile, then run
1) Just require the gem, and create a `FogTracker::Tracker`. Pass it some account information in a hash, perhaps loaded from a YAML file:

rake db:migrate:tracker

2) In your Ruby app, `require` the gem, and set up an ActiveRecord connection. In Rails, the connection is set up for you automatically on startup, but here's an example for a non-Rails app:

require 'cloud_cost_tracker'
ActiveRecord::Base.establish_connection({
:adapter => 'sqlite3', :database => 'cloud_cost_tracker.sqlite3'
})

3) Track all accounts loaded from a YAML file (or the Hash equivalent):

tracker = CloudCostTracker::Tracker.new(YAML::load(File.read 'accounts.yml'))
require 'fog_tracker'
tracker = FogTracker::Tracker.new(YAML::load(File.read 'accounts.yml'))
tracker.start

For the accounts file format, see the example below or the included `config/accounts.yml.example`.


----------------
Usage [from the command line]
----------------
1) First, generate an ActiveRecord-style database configuration file.
Here are the contents of a sample `database.yml`:

adapter: sqlite3
database: cloud_cost_tracker.sqlite3
pool: 5
timeout: 5000

If necessary, create the database to contain the data. The BillingRecords table will be created / updated by an ActiveRecord Migration.

2) Generate a YAML file containing your Fog accounts and their credentials:
Here are the contents of a sample `accounts.yml`:
Here are the contents of a sample `accounts.yml`:

AWS EC2 development account:
:provider: AWS
Expand All @@ -72,13 +43,9 @@ Usage [from the command line]
:aws_access_key_id: XXXXXXXXXXXXXXXXXXXX
:aws_secret_access_key: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
:polling_time: 180
AWS EC2 production account:
:provider: AWS
:service: Compute
:credentials:
:aws_access_key_id: XXXXXXXXXXXXXXXXXXXX
:aws_secret_access_key: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
:polling_time: 120
:exclude_resources:
- :flavors
- :images
Rackspace development account:
:provider: Rackspace
:service: Compute
Expand All @@ -87,32 +54,30 @@ Usage [from the command line]
:rackspace_username: XXXXXXXXX
:polling_time: 180

3) Run the tracker, and point it at the both the database config file and the accounts file.
2) The tracker will run asynchronously. You can call `start()` and `stop()` on it, and query the resulting collections of Fog Resource objects in several ways:

tracker database.yml accounts.yml --migrate
# get all Compute instances across all accounts and providers
tracker.get(:service => "Compute", :type => "server")

The `--migrate` argument updates the database to the latest version of the schema, and is only necessary for new databases, or when upgrading to a new version of the Tracker gem.
# get all Amazon Web Services Compute Instances across all accounts
tracker.get(:service => "Compute", :type => "server", :provider => "AWS")


----------------
Development
Who is it? (Contribution / Development)
----------------
This project is still in its early stages, but much of the framework is in place. More resource costs need to be modeled, but the patterns for the code to do so are now laid out. Helping hands are appreciated!
This project is still in its early stages, and needs to be tested with many more of Fog's cloud providers. Helping hands are appreciated!

1) Install project dependencies.

gem install rake bundler

2) Fetch the project code and bundle up...

git clone https://github.com/benton/cloud_cost_tracker.git
cd cloud_cost_tracker
git clone https://github.com/benton/fog_tracker.git
cd fog_tracker
bundle

3) Create a SQLite database for development

rake db:migrate:tracker

4) Run the tests:
3) Run the tests:

rake
47 changes: 12 additions & 35 deletions bin/tracker
Original file line number Diff line number Diff line change
@@ -1,22 +1,21 @@
#!/usr/bin/env ruby

# == Synopsis
# Uses the Fog gem to generate billing records for cloud computing resources
# Uses the Fog gem to track the status of cloud computing resources
#
# == Usage
# tracker.rb [options] DB_CONFIG_FILE.YML ACCOUNTS_CONFIG_FILE.YML
# tracker.rb [options] ACCOUNTS_CONFIG_FILE.YML
#
# == Options (all options can be put into the config file)
# -d, --delay [INTEGER] Seconds between status updates. default = 180
# -l, --log-level [LEVEL] Sets Log4r level for console output. default = INFO
# -m, --migrate Update database schema
# -h, --help Displays help message
#
# == Author
# Benton Roberts

require 'optparse'
require 'cloud_cost_tracker'
require 'fog_tracker'
LOG_LVLS = {
"DEBUG" => ::Logger::DEBUG,
"INFO" => ::Logger::INFO,
Expand All @@ -25,28 +24,19 @@ LOG_LVLS = {
"FATAL" => ::Logger::FATAL
}

module CloudCostTracker
class CloudCostTrackerConsoleApp
module FogTracker
class FogTrackerConsoleApp

def initialize
@log = CloudCostTracker.default_logger
@log = FogTracker.default_logger
parse_options
@log.info "Loading database configuration from #{ARGV[0]}"
connect_to_database(YAML::load(File.open(ARGV[0])))
@log.info "Loading account information from #{ARGV[1]}"
@accounts = YAML::load(File.open(ARGV[1]))
@tracker = CloudCostTracker::Tracker.new(
@log.info "Loading account information from #{ARGV[0]}"
@accounts = YAML::load(File.open(ARGV[0]))
@tracker = FogTracker::Tracker.new(
@accounts, {:logger => @log, :delay => @opts[:delay]}
)
end

def connect_to_database(db_config)
@log.info "Connecting to database #{db_config['database']}"
require 'active_record'
::ActiveRecord::Base.establish_connection(db_config)
migrate if @opts[:migrate]
end

def go
@tracker.start
while true do
Expand All @@ -57,40 +47,27 @@ module CloudCostTracker
def parse_options
@opts = {:log_level => 'INFO'}
optparse = OptionParser.new do |opts|
opts.banner = "Usage: tracker [options] DB_CONFIG.YML ACCOUNTS.YML"
opts.banner = "Usage: tracker [options] ACCOUNTS.YML"
opts.on('-d', '--delay SECONDS', Integer,
'Number of seconds between status updates') do |delay|
@opts[:delay] = delay
end
opts.on('-l', '--log-level LEVEL', 'Set logging level') do |log_level|
@opts[:log_level] = log_level.upcase
end
opts.on('-m', '--migrate', 'Update database schema') do
@opts[:migrate] = true
end
opts.on('-h', '--help', 'Display this help message') do
puts opts ; exit
end
end
optparse.parse!
@log.level = LOG_LVLS[@opts[:log_level]] if LOG_LVLS[@opts[:log_level]]
if ARGV.count < 1
@log.error "A database config file must be specified"
exit 1
end
if ARGV.count < 2
@log.error "A YAML file with account info must be specified"
exit 2
exit 1
end
end

def migrate
@log.info "Updating database schema..."
migration_dir = File.expand_path('../../db/migrate', __FILE__)
ActiveRecord::Migrator.migrate migration_dir
end

end
end

CloudCostTracker::CloudCostTrackerConsoleApp.new.go
FogTracker::FogTrackerConsoleApp.new.go
33 changes: 0 additions & 33 deletions cloud_cost_tracker.gemspec

This file was deleted.

12 changes: 8 additions & 4 deletions config/accounts.yml.example
Original file line number Diff line number Diff line change
@@ -1,17 +1,21 @@
AWS EC2 development account:
AWS EC2 production account:
:provider: AWS
:service: Compute
:credentials:
:aws_access_key_id: XXXXXXXXXXXXXXXXXXXX
:aws_secret_access_key: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
:polling_time: 180
AWS EC2 production account:
:polling_time: 120
:exclude_resources:
AWS EC2 development account:
:provider: AWS
:service: Compute
:credentials:
:aws_access_key_id: XXXXXXXXXXXXXXXXXXXX
:aws_secret_access_key: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
:polling_time: 120
:polling_time: 180
:exclude_resources:
- :flavors
- :images
Rackspace development account:
:provider: Rackspace
:service: Compute
Expand Down
4 changes: 0 additions & 4 deletions config/database.yml

This file was deleted.

22 changes: 0 additions & 22 deletions db/migrate/20120118000000_create_billing_records.rb

This file was deleted.

31 changes: 31 additions & 0 deletions fog_tracker.gemspec
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
# -*- encoding: utf-8 -*-
$:.push File.expand_path("../lib", __FILE__)
require "fog_tracker/version"

Gem::Specification.new do |s|
s.name = "fog_tracker"
s.version = FogTracker::VERSION
s.authors = ["Benton Roberts"]
s.email = ["[email protected]"]
s.homepage = "http://github.com/benton/fog_tracker"
s.summary = %q{Tracks the state of cloud computing resources across }+
%q{multiple accounts with multiple service providers}
s.description = %q{This gem peridically polls mutiple cloud computing }+
%q{services using the fog gem, asynchronously updating the }+
%q{state of the resulting collections of Fog Resources.}

s.rubyforge_project = "fog_tracker"

s.files = `git ls-files`.split("\n")
s.test_files = `git ls-files -- {test,spec,features}/*`.split("\n")
s.executables = `git ls-files -- bin/*`.split("\n").map{ |f| File.basename(f) }
s.require_paths = ["lib"]

# Runtime dependencies
s.add_dependency "fog"

# Development / Test dependencies
s.add_development_dependency "rake"
s.add_development_dependency "rspec"
s.add_development_dependency "rdoc"
end
Loading

0 comments on commit 8274dcc

Please sign in to comment.