Monday, May 31, 2010

youtube-dl: Download videos from YouTube.com

What is it?

youtube-dl is a small command-line program to download videos from YouTube.com. It requires the Python interpreter, version 2.x (x being at least 4), and it's not platform specific. It should work in your Unix box, in Windows or in Mac OS X. The latest version is 2010.04.04. It's released to the public domain, which means you can modify it, redistribute it or use it however you like.
I'll try to keep it updated if YouTube.com changes the way you access their videos. After all, it's a simple and short program. However, I can't guarantee anything. If you detect it stops working, check for new versions and/or inform me about the problem, indicating the program version you are using. If the program stops working and I can't solve the problem but you have a solution, I'd like to know it. If that happens and you feel you can maintain the program yourself, tell me.ok.

Usage instructions

In Windows, once you have installed the Python interpreter, save the program with the .py extension and put it somewhere in the PATH. Try to follow the guide to install youtube-dl under Windows XP.
In Unix, download it, give it execution permission and copy it to one of the PATH directories (typically, /usr/local/bin).
After that, you should be able to call it from the command line as youtube-dl or youtube-dl.py. I will use youtube-dl in the following examples. Usage instructions are easy. Use youtube-dl followed by a video URL or identifier. Example: youtube-dl "http://www.youtube.com/watch?v=foobar". The video will be saved to the file foobar.flv in that example. As YouTube.com videos are in Flash Video format, their extension should be flv. In Linux and other unices, video players using a recent version of ffmpeg can play them. That includes MPlayer, VLC, etc. Those two work under Windows and other platforms, but you could also get a specific FLV player of your taste.
If you try to run the program and you receive an error message containing the keyword SyntaxError near the end, it means your Python interpreter is too old.

More usage tips

  • You can change the file name of the video using the -o option, like in youtube-dl -o vid.flv "http://www.youtube.com/watch?v=foobar". Read the Output template section for more details on this.
  • Some videos require an account to be downloaded, mostly because they're flagged as mature content. You can pass the program a username and password for a YouTube.com account with the -u and -p options, like youtube-dl -u myusername -p mypassword "http://www.youtube.com/watch?v=foobar".
  • The account data can also be read from the user .netrc file by indicating the -n or --netrc option. The machine name is youtube in that case.
  • The simulate mode (activated with -s or --simulate) can be used to just get the real video URL and use it with a download manager if you prefer that option.
  • The quiet mode (activated with -q or --quiet) can be used to supress all output messages. This allows, in systems featuring /dev/stdout and other similar special files, outputting the video data to standard output in order to pipe it to another program without interferences.
  • The program can be told to simply print the final video URL to standard output using the -g or --get-url option.
  • In a similar line, the -e or --get-title option tells the program to print the video title.
  • The default filename is video_id.flv. But you can also use the video title in the filename with the -t or --title option, or preserve the literal title in the filename with the -l or --literal option.
  • You can make the program append &fmt=something to the URL by using the -f or --format option. This makes it possible to download high quality versions of the videos when available.
  • The -b or --best-quality option can be used to download the highest available quality version of any given video.
  • The -m or --mobile-version option is an alias for -f 17.
  • The -d or --high-def option is an alias for -f 22.
  • Normally, the program will stop on the first error, but you can tell it to attempt to download every video with the -i or --ignore-errors option.
  • The -a or --batch-file option lets you specify a file to read URLs from. The file must contain one URL per line.
  • The program can be told not to overwrite existing files using the -w or --no-overwrites option.
  • It can be told to attempt to continue interrupted downloads with the -c or --continue option.
  • For YouTube, you can also use the URL of a playlist, and it will download all the videos in that playlist.
  • For YouTube, you can also use the special word ytsearch to download search results. With ytsearch it will download the first search result. With ytsearchN, where N is a number, it will download the first N results. With ytsearchall it will download every result for that search. In most systems you'll need to use quotes for multiple words. Example: youtube-dl "ytsearch3:cute kittens".
  • youtube-dl honors the http_proxy environment variable if you want to use a proxy. Set it to something like http://proxy.example.com:8080, and do not leave the http:// prefix out.
  • You can get the program version by calling it as youtube-dl -v or youtube-dl --version.
  • For usage instructions, use youtube-dl -h or youtube-dl --help.
  • You can cancel the program at any time pressing Ctrl+C. It may print some error lines saying something about KeyboardInterrupt. That's ok. 

Output template

The -o option allows users to indicate a template for the output file names. The basic usage is not to set any template arguments when downloading a single file, like in youtube-dl -o funny_video.flv "http://some/video". However, it may contain special sequences that will be replaced when downloading each video. The special sequences have the format %(NAME)s. To clarify, that's a percent symbol followed by a name in parenthesis, followed by a lowercase S. Allowed names are:
  • id: The sequence will be replaced by the video identifier.
  • url: The sequence will be replaced by the video URL.
  • uploader: The sequence will be replaced by the nickname of the person who uploaded the video.
  • title: The sequence will be replaced by the literal video title.
  • stitle: The sequence will be replaced by a simplified video title, restricted to alphanumeric characters and dashes.
  • ext: The sequence will be replaced by the appropriate extension (like flv or mp4).
  • epoch: The sequence will be replaced by the Unix epoch when creating the file.
  • ord: The sequence will be replaced by a five-digit number that will be increased with each download, starting at zero.
As you may have guessed, the default template is %(id)s.%(ext)s. When some command line options are used, it's replaced by other templates like %(title)s-%(id)s.%(ext)s. You can specify your own.

 


 

Tuesday, May 11, 2010

How to Format a Hard Drive in Linux Command Line

Preparation

1)When you format a computer hard drive, you will lose everything that is on the drive. Therefore, it is very important to back up anything you might later want.

2)To format a secondary drive, you will need root access. All commands used in this tutorial are run from a command line (terminal) as root. You may also want to make note of the size (total capacity in MB) of the drive.

3)Linux refers to hard drive as either "hdx" or "sdx," where x is a letter, starting with a, which represents the order in which the drive was added to or detected by the computer. The "hd" prefix is used for IDE and PATA (formerly just ATA), and the "sd" prefix is used for SCSI, SATA and USB drives. Usually, a number is also put at the end of "hdx" or "sdx" to denote different partitions on the same physical drive, but for the purpose of formatting, you only need to know which letter the drive you want to format is.

4)The examples given in this how-to are for a computer with two IDE hard drives attached as a master and slave. In this, the drives are "hda" and "hdb." You will need to determine the letter of the drive you want to format for your own setup. We will be formatting the drive hdb. For all examples, replace "hdb" with whatever your drive designation is.

5)You can see all the drives attached to your system by typing the command "ls /dev/hd*" or "ls /dev/sd*", depending on which type (IDE, SATA and so forth) the drives are. On the example system, the result of this command looks like "/dev/hda /dev/hda1 /dev/hda2 /dev/hdb /dev/hdb1". The operating system is installed on hda, which has two partitions (hda1 and hda2), and there is one partition on hdb and hdb1.

Using fdisk

1)First, you will use the fdisk command to erase any old partitions on the drive and create a new one. Any changes you make using fdisk are only made permanent if you then issue the "w" command before quitting, so feel free to play around a little if you like. If at any time you find yourself stuck, you can quit the program without saving changes by holding the "Ctrl" key and pressing "c."

2)At the command prompt, type "fdisk /dev/hdb", replacing the "hdb" with the letters for your drive. Upon opening, fdisk may give you a couple of warnings, all of which can be ignored. It then gives you a prompt that looks like this: Command (m for help):

3)Enter "p" to see the partition table of the drive. The first line of output from the "p" command will also tell you the size of the drive. This is a good way to double-check that you are working with the correct drive.
4)If there are any partitions already on the drive, they will be listed as the last lines of the "p" command. On our example, this looks like "/dev/hdb1", followed by some information about the partition's size and filesystem.

5)To delete any existing partitions, press "d" and then "Enter." It will ask you which partition number you wish to delete. The number of the partition is the number that follows hdb, so on our example system, we enter 1. If there are multiple partitions, repeat the "d" command for each one. You can always view the partition table again with the "p" command.

6)Once you have deleted all existing partitions on the drive, you are ready to make a new one. Type "n" and hit "Enter." Then press "p" to create a primary partition. It asks you for a partition number; enter "1." Now you are asked which cylinder the partition should start at. The beginning of the drive is the default, so just hit "Enter." Then, you are asked for the last cylinder. The end of the drive is the default, so you can just press "Enter" again.

7)Now you are back at fdisk's command prompt. Use the "p" command to check the partition table. You should now see your new partition at the bottom of the output. In the example, it lists as "/dev/hdb1."

8)You now need to set the filesystem type for your new partition with the "t" command. You are asked for the Hex code of the filesystem you wish to use. We will use the standard Linux ext2 filesystem, which is "83." If you are doing something special and know of a particular filesystem that you need to use, you can press "L" to see all the codes, which are one or two characters made up of the numbers 0 to 9 and the letters a to f.

9)Now just issue the "w" command to write your new partition table and exit fdisk.

Creating the New Filesystem With mkfs

1)Now you need to create the filesystem on the drive. This is done with the "mkfs" command.

2)At the command prompt, enter "mkfs -t ext2 /dev/hdb1", while remembering to change the hdb1 to whatever the letters are for the partition you just created.

3)If you are using a different filesystem than ext2, you will have to specify that where "ext2" is in the above command.

Finalizing the Format With fsck

1)All that is left is to run a check on the drive and enter it into your fstab so that the drive mounts each time you start your computer. This can be done with a single fsck command.

2)At the command prompt, type "fsck -f -y /dev/hdb1", again replacing hdb1 with the letters and number for your partition.

3)After fsck runs, your new drive is formatted. Restart your system before using it.

4)If you reformatted your system drive, you will now need to boot off an installation disk to install an operating system.

Monday, May 3, 2010

Setting Up A Samba Server with Windows XP Clients

First Setting up the Smb Server:

1) For setting up samba server configure smb.conf file which is usually found under /etc/samba/ directory.

Notes: After changing smb.conf always restart the samba server

Commands to be used:
./smbd stop
./smbd start

2) Changes to be done in the smb.conf file are
Notes:
a) Whenever you modify this file you should run the command "testparm" to check that you have not made any basic syntactic errors.  
b) Read the smb.conf man page to understand the various parts of the file

In the Global section [global] the significant entries are:

[global]
logon drive = p: // Here you can set the drive letter
encrypt passwords = yes

/* To allow only specific hosts who can connect to server below entry is used make sure localhost is added */

hosts allow = IP-address, 127.0.0.1/255.255.255.0

logon home = \\netbiosname\%U // home directories will be mounted as the user logs on.

passwd program = /usr/bin/passwd %u

wins support = yes //Enable the windows support
dns proxy = no

netbios name = wealhtheow //Set the netbiosname

netbios aliases = rch01 //Set the alias, this is optional

unix password sync = No //Sync the unix passwords
(Samba (actually 'smbd') makes use of the file 'smb.conf' to know about its configuration. During installation of samba, smb.conf is created in the directory '/etc/samba' by default. But in FreeBSD all the system configuration files are put in the directory /usr/local/etc/. There were two smb.conf files that were existing on this system and therefore a link was created from /etc/samba/smb.conf to the actual file used by FreeBSD which is /usr/local/etc/smb.conf. This was done to make sure that the Samba server uses the correct version of the configuration file. To resolve the passwords problem, the smb.conf file was modified to remove synchronization between the Samba and Unix passwords.)

logon path = \\rch01\profiles\%U // This will tell the server where to place the home directories

workgroup = rch.uky.edu // Workgroup to be used, all clients should have the same workgroup

os level = 64 // set the OS level

// Below entries are important in order to tell that Samba server is the domain controller
domain logons = yes
pam password change = yes
preferred master = yes
domain master = yes
local master = yes


In the [homes] section the following entries are important :

[homes]
comment = Home Directories
browseable = no
writable = yes
valid users = %S
read only = no
create mode = 0600
directory mode = 0700

logon home = \\rch01\%U // This tells where is the home directory for the user

hide files = /*.pst/


In the [profiles] section the path variable tells where the profiles are to be placed and make sure the directories are created with right names

[profiles]
create mode = 0600
directory mode = 0700
path = /home/samba/profiles/
profile acls = yes
read only = no
writable = yes


Configuring the Windows XP Client:


Notes : Only Windows XP-Professional Edition can join the Domain, it does not work for WindowsXP-Home Edition.

STEPS:

1) Make sure that the workstation belonged to the same workgroup as the server and have a fixed IP address and hostname assigned.

2) Change the registry entry, run the command regedt32 and do the below
a) RequireSignOrSeal Registry hack

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\netlogon\parameters
"RequireSignOrSeal"=dword:00000000

b) Use the Registry Editor and edit the
HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows\System\CompatibleRUPSecurity to have the DWORD value of 1

3) Use the Group Policy Editor (gpedit.msc) and enable "Computer Configuration\Administrative Templates\System\User Profiles\Do not check for user ownership of Roaming Profile Folders".

4) Go to MyComputer right click Properties. Go to Change and click on Domain and enter the domain-name you want to join. When joining the domain for the First time enter userid as root and give the samba password. Make sure there is an entry for the root in the smbpasswd (samba password) file.

5) Reboot and then the changes will be effective.