Vkontakte html & parse
This commit is contained in:
parent
2e762a7674
commit
1494e81fa7
1
Gemfile
1
Gemfile
|
@ -9,6 +9,7 @@ gem 'pg'
|
||||||
gem 'mongo', '1.3.0'
|
gem 'mongo', '1.3.0'
|
||||||
gem 'bson_ext', '1.3.0'
|
gem 'bson_ext', '1.3.0'
|
||||||
gem 'mongoid', '~> 2.0'
|
gem 'mongoid', '~> 2.0'
|
||||||
|
gem 'nokogiri'
|
||||||
|
|
||||||
# Use unicorn as the web server
|
# Use unicorn as the web server
|
||||||
# gem 'unicorn'
|
# gem 'unicorn'
|
||||||
|
|
|
@ -48,6 +48,7 @@ GEM
|
||||||
mongo (~> 1.3)
|
mongo (~> 1.3)
|
||||||
tzinfo (~> 0.3.22)
|
tzinfo (~> 0.3.22)
|
||||||
will_paginate (~> 3.0.pre)
|
will_paginate (~> 3.0.pre)
|
||||||
|
nokogiri (1.4.4)
|
||||||
pg (0.10.1)
|
pg (0.10.1)
|
||||||
polyglot (0.3.1)
|
polyglot (0.3.1)
|
||||||
rack (1.2.2)
|
rack (1.2.2)
|
||||||
|
@ -82,5 +83,6 @@ DEPENDENCIES
|
||||||
bson_ext (= 1.3.0)
|
bson_ext (= 1.3.0)
|
||||||
mongo (= 1.3.0)
|
mongo (= 1.3.0)
|
||||||
mongoid (~> 2.0)
|
mongoid (~> 2.0)
|
||||||
|
nokogiri
|
||||||
pg
|
pg
|
||||||
rails (= 3.0.5)
|
rails (= 3.0.5)
|
||||||
|
|
257
README
257
README
|
@ -1,256 +1 @@
|
||||||
== Welcome to Rails
|
Gouranga gives a fuck
|
||||||
|
|
||||||
Rails is a web-application framework that includes everything needed to create
|
|
||||||
database-backed web applications according to the Model-View-Control pattern.
|
|
||||||
|
|
||||||
This pattern splits the view (also called the presentation) into "dumb"
|
|
||||||
templates that are primarily responsible for inserting pre-built data in between
|
|
||||||
HTML tags. The model contains the "smart" domain objects (such as Account,
|
|
||||||
Product, Person, Post) that holds all the business logic and knows how to
|
|
||||||
persist themselves to a database. The controller handles the incoming requests
|
|
||||||
(such as Save New Account, Update Product, Show Post) by manipulating the model
|
|
||||||
and directing data to the view.
|
|
||||||
|
|
||||||
In Rails, the model is handled by what's called an object-relational mapping
|
|
||||||
layer entitled Active Record. This layer allows you to present the data from
|
|
||||||
database rows as objects and embellish these data objects with business logic
|
|
||||||
methods. You can read more about Active Record in
|
|
||||||
link:files/vendor/rails/activerecord/README.html.
|
|
||||||
|
|
||||||
The controller and view are handled by the Action Pack, which handles both
|
|
||||||
layers by its two parts: Action View and Action Controller. These two layers
|
|
||||||
are bundled in a single package due to their heavy interdependence. This is
|
|
||||||
unlike the relationship between the Active Record and Action Pack that is much
|
|
||||||
more separate. Each of these packages can be used independently outside of
|
|
||||||
Rails. You can read more about Action Pack in
|
|
||||||
link:files/vendor/rails/actionpack/README.html.
|
|
||||||
|
|
||||||
|
|
||||||
== Getting Started
|
|
||||||
|
|
||||||
1. At the command prompt, create a new Rails application:
|
|
||||||
<tt>rails new myapp</tt> (where <tt>myapp</tt> is the application name)
|
|
||||||
|
|
||||||
2. Change directory to <tt>myapp</tt> and start the web server:
|
|
||||||
<tt>cd myapp; rails server</tt> (run with --help for options)
|
|
||||||
|
|
||||||
3. Go to http://localhost:3000/ and you'll see:
|
|
||||||
"Welcome aboard: You're riding Ruby on Rails!"
|
|
||||||
|
|
||||||
4. Follow the guidelines to start developing your application. You can find
|
|
||||||
the following resources handy:
|
|
||||||
|
|
||||||
* The Getting Started Guide: http://guides.rubyonrails.org/getting_started.html
|
|
||||||
* Ruby on Rails Tutorial Book: http://www.railstutorial.org/
|
|
||||||
|
|
||||||
|
|
||||||
== Debugging Rails
|
|
||||||
|
|
||||||
Sometimes your application goes wrong. Fortunately there are a lot of tools that
|
|
||||||
will help you debug it and get it back on the rails.
|
|
||||||
|
|
||||||
First area to check is the application log files. Have "tail -f" commands
|
|
||||||
running on the server.log and development.log. Rails will automatically display
|
|
||||||
debugging and runtime information to these files. Debugging info will also be
|
|
||||||
shown in the browser on requests from 127.0.0.1.
|
|
||||||
|
|
||||||
You can also log your own messages directly into the log file from your code
|
|
||||||
using the Ruby logger class from inside your controllers. Example:
|
|
||||||
|
|
||||||
class WeblogController < ActionController::Base
|
|
||||||
def destroy
|
|
||||||
@weblog = Weblog.find(params[:id])
|
|
||||||
@weblog.destroy
|
|
||||||
logger.info("#{Time.now} Destroyed Weblog ID ##{@weblog.id}!")
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
The result will be a message in your log file along the lines of:
|
|
||||||
|
|
||||||
Mon Oct 08 14:22:29 +1000 2007 Destroyed Weblog ID #1!
|
|
||||||
|
|
||||||
More information on how to use the logger is at http://www.ruby-doc.org/core/
|
|
||||||
|
|
||||||
Also, Ruby documentation can be found at http://www.ruby-lang.org/. There are
|
|
||||||
several books available online as well:
|
|
||||||
|
|
||||||
* Programming Ruby: http://www.ruby-doc.org/docs/ProgrammingRuby/ (Pickaxe)
|
|
||||||
* Learn to Program: http://pine.fm/LearnToProgram/ (a beginners guide)
|
|
||||||
|
|
||||||
These two books will bring you up to speed on the Ruby language and also on
|
|
||||||
programming in general.
|
|
||||||
|
|
||||||
|
|
||||||
== Debugger
|
|
||||||
|
|
||||||
Debugger support is available through the debugger command when you start your
|
|
||||||
Mongrel or WEBrick server with --debugger. This means that you can break out of
|
|
||||||
execution at any point in the code, investigate and change the model, and then,
|
|
||||||
resume execution! You need to install ruby-debug to run the server in debugging
|
|
||||||
mode. With gems, use <tt>sudo gem install ruby-debug</tt>. Example:
|
|
||||||
|
|
||||||
class WeblogController < ActionController::Base
|
|
||||||
def index
|
|
||||||
@posts = Post.find(:all)
|
|
||||||
debugger
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
So the controller will accept the action, run the first line, then present you
|
|
||||||
with a IRB prompt in the server window. Here you can do things like:
|
|
||||||
|
|
||||||
>> @posts.inspect
|
|
||||||
=> "[#<Post:0x14a6be8
|
|
||||||
@attributes={"title"=>nil, "body"=>nil, "id"=>"1"}>,
|
|
||||||
#<Post:0x14a6620
|
|
||||||
@attributes={"title"=>"Rails", "body"=>"Only ten..", "id"=>"2"}>]"
|
|
||||||
>> @posts.first.title = "hello from a debugger"
|
|
||||||
=> "hello from a debugger"
|
|
||||||
|
|
||||||
...and even better, you can examine how your runtime objects actually work:
|
|
||||||
|
|
||||||
>> f = @posts.first
|
|
||||||
=> #<Post:0x13630c4 @attributes={"title"=>nil, "body"=>nil, "id"=>"1"}>
|
|
||||||
>> f.
|
|
||||||
Display all 152 possibilities? (y or n)
|
|
||||||
|
|
||||||
Finally, when you're ready to resume execution, you can enter "cont".
|
|
||||||
|
|
||||||
|
|
||||||
== Console
|
|
||||||
|
|
||||||
The console is a Ruby shell, which allows you to interact with your
|
|
||||||
application's domain model. Here you'll have all parts of the application
|
|
||||||
configured, just like it is when the application is running. You can inspect
|
|
||||||
domain models, change values, and save to the database. Starting the script
|
|
||||||
without arguments will launch it in the development environment.
|
|
||||||
|
|
||||||
To start the console, run <tt>rails console</tt> from the application
|
|
||||||
directory.
|
|
||||||
|
|
||||||
Options:
|
|
||||||
|
|
||||||
* Passing the <tt>-s, --sandbox</tt> argument will rollback any modifications
|
|
||||||
made to the database.
|
|
||||||
* Passing an environment name as an argument will load the corresponding
|
|
||||||
environment. Example: <tt>rails console production</tt>.
|
|
||||||
|
|
||||||
To reload your controllers and models after launching the console run
|
|
||||||
<tt>reload!</tt>
|
|
||||||
|
|
||||||
More information about irb can be found at:
|
|
||||||
link:http://www.rubycentral.com/pickaxe/irb.html
|
|
||||||
|
|
||||||
|
|
||||||
== dbconsole
|
|
||||||
|
|
||||||
You can go to the command line of your database directly through <tt>rails
|
|
||||||
dbconsole</tt>. You would be connected to the database with the credentials
|
|
||||||
defined in database.yml. Starting the script without arguments will connect you
|
|
||||||
to the development database. Passing an argument will connect you to a different
|
|
||||||
database, like <tt>rails dbconsole production</tt>. Currently works for MySQL,
|
|
||||||
PostgreSQL and SQLite 3.
|
|
||||||
|
|
||||||
== Description of Contents
|
|
||||||
|
|
||||||
The default directory structure of a generated Ruby on Rails application:
|
|
||||||
|
|
||||||
|-- app
|
|
||||||
| |-- controllers
|
|
||||||
| |-- helpers
|
|
||||||
| |-- mailers
|
|
||||||
| |-- models
|
|
||||||
| `-- views
|
|
||||||
| `-- layouts
|
|
||||||
|-- config
|
|
||||||
| |-- environments
|
|
||||||
| |-- initializers
|
|
||||||
| `-- locales
|
|
||||||
|-- db
|
|
||||||
|-- doc
|
|
||||||
|-- lib
|
|
||||||
| `-- tasks
|
|
||||||
|-- log
|
|
||||||
|-- public
|
|
||||||
| |-- images
|
|
||||||
| |-- javascripts
|
|
||||||
| `-- stylesheets
|
|
||||||
|-- script
|
|
||||||
|-- test
|
|
||||||
| |-- fixtures
|
|
||||||
| |-- functional
|
|
||||||
| |-- integration
|
|
||||||
| |-- performance
|
|
||||||
| `-- unit
|
|
||||||
|-- tmp
|
|
||||||
| |-- cache
|
|
||||||
| |-- pids
|
|
||||||
| |-- sessions
|
|
||||||
| `-- sockets
|
|
||||||
`-- vendor
|
|
||||||
`-- plugins
|
|
||||||
|
|
||||||
app
|
|
||||||
Holds all the code that's specific to this particular application.
|
|
||||||
|
|
||||||
app/controllers
|
|
||||||
Holds controllers that should be named like weblogs_controller.rb for
|
|
||||||
automated URL mapping. All controllers should descend from
|
|
||||||
ApplicationController which itself descends from ActionController::Base.
|
|
||||||
|
|
||||||
app/models
|
|
||||||
Holds models that should be named like post.rb. Models descend from
|
|
||||||
ActiveRecord::Base by default.
|
|
||||||
|
|
||||||
app/views
|
|
||||||
Holds the template files for the view that should be named like
|
|
||||||
weblogs/index.html.erb for the WeblogsController#index action. All views use
|
|
||||||
eRuby syntax by default.
|
|
||||||
|
|
||||||
app/views/layouts
|
|
||||||
Holds the template files for layouts to be used with views. This models the
|
|
||||||
common header/footer method of wrapping views. In your views, define a layout
|
|
||||||
using the <tt>layout :default</tt> and create a file named default.html.erb.
|
|
||||||
Inside default.html.erb, call <% yield %> to render the view using this
|
|
||||||
layout.
|
|
||||||
|
|
||||||
app/helpers
|
|
||||||
Holds view helpers that should be named like weblogs_helper.rb. These are
|
|
||||||
generated for you automatically when using generators for controllers.
|
|
||||||
Helpers can be used to wrap functionality for your views into methods.
|
|
||||||
|
|
||||||
config
|
|
||||||
Configuration files for the Rails environment, the routing map, the database,
|
|
||||||
and other dependencies.
|
|
||||||
|
|
||||||
db
|
|
||||||
Contains the database schema in schema.rb. db/migrate contains all the
|
|
||||||
sequence of Migrations for your schema.
|
|
||||||
|
|
||||||
doc
|
|
||||||
This directory is where your application documentation will be stored when
|
|
||||||
generated using <tt>rake doc:app</tt>
|
|
||||||
|
|
||||||
lib
|
|
||||||
Application specific libraries. Basically, any kind of custom code that
|
|
||||||
doesn't belong under controllers, models, or helpers. This directory is in
|
|
||||||
the load path.
|
|
||||||
|
|
||||||
public
|
|
||||||
The directory available for the web server. Contains subdirectories for
|
|
||||||
images, stylesheets, and javascripts. Also contains the dispatchers and the
|
|
||||||
default HTML files. This should be set as the DOCUMENT_ROOT of your web
|
|
||||||
server.
|
|
||||||
|
|
||||||
script
|
|
||||||
Helper scripts for automation and generation.
|
|
||||||
|
|
||||||
test
|
|
||||||
Unit and functional tests along with fixtures. When using the rails generate
|
|
||||||
command, template test files will be generated for you and placed in this
|
|
||||||
directory.
|
|
||||||
|
|
||||||
vendor
|
|
||||||
External libraries that the application depends on. Also includes the plugins
|
|
||||||
subdirectory. If the app has frozen rails, those gems also go here, under
|
|
||||||
vendor/rails/. This directory is in the load path.
|
|
|
@ -8,6 +8,7 @@ class ArtistController < ApplicationController
|
||||||
name = params[:name].gsub('+', ' ').gsub('%20', ' ')
|
name = params[:name].gsub('+', ' ').gsub('%20', ' ')
|
||||||
end
|
end
|
||||||
@artist = Artist.getByName(name)
|
@artist = Artist.getByName(name)
|
||||||
|
pp @artist
|
||||||
end
|
end
|
||||||
end
|
end
|
||||||
|
|
||||||
|
|
|
@ -2,6 +2,12 @@ class TrackController < ApplicationController
|
||||||
require 'net/http'
|
require 'net/http'
|
||||||
require 'uri'
|
require 'uri'
|
||||||
def listen
|
def listen
|
||||||
|
artist = 'The White Stripes'
|
||||||
|
track = 'Blue Orchid'
|
||||||
|
length = 157
|
||||||
|
|
||||||
|
Vkontakte.get(artist, track, length)
|
||||||
|
|
||||||
|
render :json => 0
|
||||||
end
|
end
|
||||||
end
|
end
|
||||||
|
|
|
@ -1,19 +1,70 @@
|
||||||
module Vkontakte
|
module Vkontakte
|
||||||
@@accounts = YAML.load_file("#{RAILS_ROOT}/config/vk_accounts.yml")
|
require 'net/http'
|
||||||
|
require 'erb'
|
||||||
|
#require 'nokogiri'
|
||||||
|
|
||||||
def get artist, track, length
|
@@accounts = YAML.load_file("#{Rails.root.to_s}/config/vk_accounts.yml")
|
||||||
|
|
||||||
|
def self.get artist, track, length
|
||||||
|
#html = self.getHtml artist +' - '+ track
|
||||||
|
files = self.parseHtml 1 #html
|
||||||
|
files = self.calcWeight files
|
||||||
end
|
end
|
||||||
|
|
||||||
def getHtml q
|
|
||||||
|
|
||||||
|
private
|
||||||
|
|
||||||
|
def self.randomBot
|
||||||
|
@@accounts[@@accounts.keys[rand(@@accounts.keys.length - 1)]]
|
||||||
end
|
end
|
||||||
|
|
||||||
def parseHtml html
|
def self.getHtml q
|
||||||
|
bot = self.randomBot
|
||||||
|
headers = {
|
||||||
|
'Cookie' => 'remixsid='+ bot['remixsid'] + ';remixchk='+ bot['remixchk'].to_s,
|
||||||
|
'Referer' => 'http://vkontakte.ru/audio?album_id=0',
|
||||||
|
'X-Requested-With' => 'XMLHttpRequest',
|
||||||
|
'Origin' => 'http://vkontakte.ru',
|
||||||
|
'Content-Type' => 'application/x-www-form-urlencoded',
|
||||||
|
'User-Agent' => bot['user_agent'],
|
||||||
|
'Connection' => 'close'
|
||||||
|
}
|
||||||
|
data = {
|
||||||
|
'act' => 'search',
|
||||||
|
'al' => 1,
|
||||||
|
'gid' => 0,
|
||||||
|
'id' => bot['user_id'],
|
||||||
|
'offset' => 0,
|
||||||
|
'q' => q,
|
||||||
|
'sort' => 2
|
||||||
|
}
|
||||||
|
data_str = []
|
||||||
|
data.each {|k, v| data_str << k +'='+ ERB::Util.url_encode(v.to_s)}
|
||||||
|
|
||||||
|
http = Net::HTTP.new('vkontakte.ru', 80)
|
||||||
|
resp, data = http.post('/audio', data_str.join('&'), headers)
|
||||||
|
pp data
|
||||||
end
|
end
|
||||||
|
|
||||||
def calcWeight files
|
def self.parseHtml html
|
||||||
|
html = open('/Users/chez/Sites/beathaven/tmp/vkres.txt').read
|
||||||
|
files = []
|
||||||
|
html
|
||||||
|
.gsub(/\\/, '')
|
||||||
|
.scan(/<table.*?<input.*?value=\"(.*?)\,\d{2,4}\".*?>.*?<div\sclass=\"duration.*?>(.*?)<\/div>.*?<div\sclass=\"title_wrap\".*?>.*?selectPerformer\(event\,\s\'(.*?)\'\).*?<span\sclass=\"title\">(.*?)<\/span><\/div>.*?<\/table>/i)
|
||||||
|
.each do |item|
|
||||||
|
t = item[1].split(':')
|
||||||
|
files << {
|
||||||
|
'url' => item[0],
|
||||||
|
'length' => (t[0].to_i * 60) + t[1].to_i,
|
||||||
|
'artist' => item[2],
|
||||||
|
'title' => item[3].gsub(/<\/?span>/, '')
|
||||||
|
}
|
||||||
|
end
|
||||||
|
files
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.calcWeight files
|
||||||
|
pp files
|
||||||
end
|
end
|
||||||
end
|
end
|
Loading…
Reference in New Issue