BeagleBone installing PCF8563 RTC

I ordered the Waveshare "Misc Cape" which uses the PCF8563 RTC chip with an I2C interface like all of them. Waveshare's online documentation is unhelpfully brief regarding how to install drivers for it. They offer a fully compiled Angstrom distribution with their drivers but since Debian is working out so well for me on my project I was hoping to try to find a way to install drivers for Debian. At the bottom of their wiki for this cape there are a bunch of links, some of which seem to be source code and other documentation, but it's not clear what it all is.

http://www.waveshare.com/wiki/MISC_CAPE

One of the links turns out to be to two SD card images, one of which is a fully compiled Debian image that has the drivers for the PCF8563 chip built in, however when I downloaded and tried it I found that although the RTC driver seemed to work well there were other issues with the distro that have to be overcome:

http://www.waveshare.com/wiki/MISC_CAPE_Image

There are other links on the same wiki page that I'm still trying to figure out. The "User Manual" link is just to a PDF of the wiki. The Schematic is helpful but really the cape is quite simple and the schematic is a single page. The Test Code link is to a zip file named XXX_CAPE_Angstrom_API.7Z which seems to contain a mix of C programs and compiled programs which don't have anything to do with the PCF8563 and furthermore seem to be for the Angstrom kernel. The section labelled "Kernel" seems to have several different source code trees for both Angstrom and Debian kernels, presumably with driver code but at the moment I still don't know how my way around kernel source code or how to compile kernels although it looks like I'm going to have to learn. The section labeled "Kernel_config" also seems to have to do with compilation so I will have to become smarter to even understand what it is. I have absolutely no idea what the link labelled "Cross-compilation toolchain" is even referring to. Under Source Code there are several links to zip files but some obviously have nothing to do with the RTC chip (like "wifi" and "mjpg-steamer") and the one link labelled "driver" is to a zip file that just has one c file with code that also doesn't seem to have anything to do with the RTC chip. Of the three links under "Documentations" the first one "Porting Drivers" actually mentions the RTC chip but the compilation instructions are like greek to me. The link for "Porting Kernel" mysteriously seems to be about building a Ubuntu kernel, and these instructions are also opaque to me at the moment.

Looking for a shortcut to figuring out the poorly documented materials on the Waveshare wiki, I started looking at the following tutorial, which showed how to interface a different chip that is on Adafruit's RTC stamp to BeagleBone.

https://learn.adafruit.com/downloads/pdf/adding-a-real-time-clock-to-beaglebone-black.pdf

Since I also bought one of those Adafruit RTC stamps, I may yet give following this tutorial as written a try, but I thought that I could adapt the steps shown to the PCF8563. I tried doing the i2cdetect command and it found the PCF8563 at its address of 0x51. I did the command to add it as a new device and that seemed to go okay. There was an rtc0 device in /dev that I was able to write a time to and read from. I went through all the rest of the steps to set up a service to read from the rtc at bootup but whenever I power cycled the BeagleBone it would come up with a default date which was March 1, 2013.

I realized that something was writing time to the rtc, and went through a bunch of research to discover that there is a hwclock.sh script that is being called by several different boot levels in Debian that seems to be setting this default time to rtc0. The hwclock.sh script is at /etc/init.d/; the directories /etc/rcS.d/, /etc/rc0.d/, and /etc/rc6.d/ all seem to have the same script also but these are really just sim links to the one in /etc/init.d/. The other directories are used to define init scripts called at different kernel boot levels. This information is all swirling around inside the following comment thread:

https://www.raspberrypi.org/forums/viewtopic.php?f=44&t=16218

I tried modifying this hwclock script as advised in the thread but didn't have any luck. I began to realize from reading the script and the messages in /var/log that the rtc0 device was some kind of default device and not the PCF8563 chip on my Waveshare cape after all. I tried doing modprobe commands as shown in the link above and in some other links I found and it seems like the rtc-pcf8563 "module" is something that I need and don't have and which should run when I add the chip as a new device on the I2C bus. I found some .ko files on my Debian system under /lib/modules/3.8.13-bone70/kernel/drivers/rtc but none of them are rtc-pcf8563.ko or rtc-ds1307.ko either for that matter.

A frequent hit that I keep getting while googling for the problems I am having installing this cape is the following site in what seems to be Hebrew or something. Without using Google Translate, I can tell that he's going through all the same steps that I did, using i2cdetect, adding it as a new device, and creating a service to load the system time from it a bootup. However, when he uses dmesg to check his log files after adding the device, he sees the "chip found" message that I'm not getting. I need to look at this site more carefully to understand what OS and distro he is using; if there's a chance he's using Debian it might give me a clue:

http://www.boonsanti.com/setting-a-real-time-clock-to-beaglebone-black-with-rtc-pcf8563/

Here's a listing for a Debian distribution that shows an rtc-pcf8563.ko file in the right place, but clearly this isn't my distribution. I need to find out the chances are that I can just find one of these .ko files and copy it onto my system (probably not great):

https://packages.debian.org/squeeze/armel/linux-image-2.6.32-5-versatile/filelist

Along the way as I did this research, I've learned that there is something called "fake-hwclock" which maintains current time in nonvolatile memory and reloads it during bootup, but I don't have that on my system. Here's a link with some info:

http://unix.stackexchange.com/questions/187261/automatically-update-hwclock-at-boot

I learned that Debian has a very nice editor called "nano" that has keyword color support and is easy to use (though not as easy as vi of course)

To edit system files, I had to change from the default login of user 'debian' to root, but when I tried "su root" it asked for a password. Various sites said that this distro had no password for root, but just hitting return on the password prompt wasn't working. I learned from the following page that I need to do "sudo su" to become root on this system:

http://www.element14.com/community/thread/32106/l/beagleboardorg-releases-debian-for-beaglebone-black?displayFullThread=true

I think that my system is using systemd instead of init. Here is what looks like nice info on systemd:

https://denibertovic.com/posts/setting-up-systemd-on-debian-in-10-minutes/

There's a book called "BeagleBone For Secret Agents" that had a google books excerpt that helped me better understand that rtc0 was not my cape, that it's an rtc inside the processor that doesn't retain time when powered off. It showed messages from a process named omap_rtc that was loading the default time and I was able to find those same messages on my system showing it setting the default clock to March 1, 2013. Here's the link to the google books excerpt:

https://books.google.com/books?id=i2OZBAAAQBAJ&pg=PT130&lpg=PT130&dq=omap_rtc+setting+system+time&source=bl&ots=9hLmjeWaPe&sig=j3ScuHRzYYN2Un2ljn_0gbsglgs&hl=en&sa=X&ved=0ahUKEwie9MXq5anKAhUIPT4KHZ6XB60Q6AEIJjAB#v=onepage&q=omap_rtc%20setting%20system%20time&f=false

Here's a blog by a guy who had a problem very similar to mine. He had an RTC chip that there was no module for in the distribution of Raspian that he had. He found drivers and compiled them and verified that the resulting module detected his chip. I need to carefully study this link to understand all of his steps and maybe try the same thing for my kernel and chip:

http://blog.remibergsma.com/2013/05/08/adding-a-hardware-clock-rtc-to-the-raspberry-pi/

When I google for drivers for the PCF8563, I usually get the following link, which interestingly is from a BeagleBone site so it seems that compiling this for BeagleBone is something that is done, although there don't seem to be any instructions or compiled module files at this link:

https://github.com/jeffegg/beaglebone/blob/master/drivers/rtc/rtc-pcf8563.c

Here's a guy interfacing a PCF8563 to a raspberry pi, and for some reason his kernel has the driver module already there. He does a modprobe command on rtc-pcf8563 and it works, so he goes merrily along with the rest of his installation steps.

http://www.susa.net/wordpress/2012/06/raspberry-pi-pcf8563-real-time-clock-rtc/

Some links to stuff that I found which didn't directly help but I'm saving just in case:

Here's a hackaday article on the PCF8563 which goes into accessing its registers and stuff without showing any kind of finished code. Interesting, but I hope I never have to use it: http://hackaday.com/2009/06/26/parts-i2c-real-time-clock-calendar-pcf8563/

Here's a raspberry pi forum where some people are digging into the source code to find out why the rtc interface for the PCF8563 doesn't work for the PI2. Not useful to me but crazy smart: https://www.raspberrypi.org/forums/viewtopic.php?f=28&t=100132

Here's setup documentation for an expansion board for the pi that uses the PCF8563. It shows the "chip found" message that I should be seeing in my logs but am not. I wish my RTC cape had this kind of documentation: https://ludwig.im/en/projects/raspberry-pi/raspberry-pi-arpi600

It turns out that Waveshare makes another RTC board with a PCF8563 for a different kind of stamp computer development system. This information wasn't helpful however because the documentation for this other RTC board included no drivers compatible with the BeagleBone: http://www.waveshare.com/pcf8563-rtc-board.htm

Debian set system time

To set the system time in Linux is very easy; the date command allows setting of the system date and time. This does not affect any time kept by a hardware clock, the hwclock command is needed for that. The BeagleBone does not come with a hardware clock, however, so the hwclock command fails out gracefully.

sudo date --set 1998-11-02
sudo date --set 21:08:00

Reference link:

https://wiki.debian.org/DateTime

BeagleBone RTC Cape

So, it turns out that the BeagleBone (or the BeagleBone Black for that matter) does not have a battery back-up for its clock, so it loses the time every time I turn it off. It is mentioned on some websites that you have to set up an NTP access to fire on bootup to keep the clock set to the current time, but of course my application is off the network. It turns out that a few people have made something called a Real Time Clock cape that has a button cell and a particular single-chip solution to act as the Beagle Board's hardware clock. Unfortunately, the nicest of these, originally sold by BeagleBoard Toys, is no longer being made.

Presently, a multipurpose board sold by Waveshare has an RTC chip and a battery. It can be bought from the company, or interestingly from Amazon.

It is also possible to either hook up the Adafruit RTC stamp to a cape protoboard as described in this article, or completely build the BeagleBoard Toys design on a protoboard.

This article shows what commands to use to set the time on the adafruit RTC stamp once the RTC cape using it is assembled.

This forum thread has more info about different RTC chips that can be used for this sort of project.

Python read file dates and rename files with datestamps

A piece of code that's so small that does so many things. Searches a directory for all files matching a pattern, gets the modification date of each file, and adds that plus a label to the file name. Uses glob to handle the searching, gets a list of matching files (may be empty if no matches, uses getmtime to get the mod time and formats it with time.strftime, uses split to dice up the original file name, and replace to slap on the label and time stamp and rejoin the new file name to the original path.

def RenameFiles(Directory, Label):

SearchString = Directory + '/pattern*'
tekfiles = glob.glob(SearchString)
for f in tekfiles:

create_date = time.strftime("%Y%m%d_%H%M%S",time.localtime(os.path.getmtime(f)))
pieces = f.split('\\')
new_name = pieces[0] + '/' + pieces[1].replace('tek','%s_pattern' % Label).replace('.','_%s.' % create_date)
os.rename(f,new_name)

Some links:

Shows use of os.rename, glob, and picking out pieces of a filename with array indexes (which I didn't use in the final code):

http://stackoverflow.com/questions/2759067/rename-files-in-python

Shows use of getctime and getmtime. It turns out that since my directory is full of copied files, that getmtime is more useful because the ctime of the copied file reflects its copy date (and can thus be later than the mtime):

http://stackoverflow.com/questions/10149994/with-python-how-to-read-the-date-created-of-a-file

http://stackoverflow.com/questions/237079/how-to-get-file-creation-modification-date-times-in-python

It took a bit of work to figure out how to take the result of getmtime and format it into a string. The result in unix epoch seconds had to be converted to a time tuple using time.localtime so that it could be formatted using time.strftime. Some of that is shown in the previous links but I also read these doc pages:

http://docs.python.org/2/library/time.html

http://epydoc.sourceforge.net/stdlib/time-module.html

Shows use of glob. I was originally thinking of making a full-featured search like my own version of 'find', but it turned out that I only needed to search specific directories, so glob did everything I really needed:

http://code.activestate.com/recipes/499305-locating-files-throughout-a-directory-tree/

http://docs.python.org/2/library/glob.html

The link which gave me the genius suggestion of using nested replace() calls to do the two filename modifications simultaneously:

http://stackoverflow.com/questions/8687018/python-string-replace-two-things-at-once

Converting TAI time

For some reason, our flight system is using "TIA". This is a French time, called either "Temps Atomique International" or International Atomic Time. It seems to be a version of UTC without leap seconds.

Here a link that supports this description of TIA. It describes TIA as 35 seconds behind UTC, but a different link seems to indicate that this has jumped another second recently. Interestingly, TIA is 19 seconds *ahead* of something called GPS time:

http://digital.ni.com/public.nsf/allkb/C5BBF6AC7036CDAA8625733000668351

This link relates it all to Unix time, which is what I really care about. Apparently there's some problem?

http://cr.yp.to/proto/utctai.html

Here's an online display of UTC and TAI. This one claims a difference of 33 seconds which must be quite old:

http://www.csgnetwork.com/timetaidispcalc.html

Here's an IDL program for converting to TAI. It has a table that relates times within ranges to the leap second differences. It goes up to July 2012:

http://cow.physics.wisc.edu/~craigm/idl/down/tai_utc.pro

It turns out that the epoch for TIA begins on January 1, 1958 (compare this to Unix time epoch of January 1, 1970):

http://books.google.com/books?id=pFO6VB_czRYC&pg=PA20&lpg=PA20&dq=tai+time+epoch&source=bl&ots=wzgSlLC2SI&sig=t-yM2y6CvwdxcIAJqREDo88uRTA&hl=en&sa=X&ei=92aKUfK-LM7xqAHZ_4CgAg&ved=0CEwQ6AEwBA#v=onepage&q=tai%20time%20epoch&f=false

The Wikipedia page for TAI covers this, although densely:

http://en.wikipedia.org/wiki/International_Atomic_Time

Here's a page from a guy that has tried to resolve unix time and TAI. He discusses several problems and their solutions:

http://www.madore.org/~david/computers/unix-leap-seconds.html

This page on timescales is just nice:

http://stjarnhimlen.se/comp/time.html

So, it looks like I need to use seconds from epoch, but compensate for the leap seconds.

How to change the windows 7 desktop clock to show 24 hour time

It's not as simple as clicking on it and finding the control for this, but it's not that far off either. This configuration has to be accessed via the control panel. Apparenly what you're doing is changing globals that apply for all software on the machine. Here's the link with nice screensnaps:

http://answers.microsoft.com/en-us/windows/forum/windows_vista-desktop/control-panel-change-12-hour-clock-to-24-hour/490a874b-7d3a-4fa4-ad38-c6ebc24a854a

Setting up Windows XP to use a time server

I needed to set up my Windows and Linux systems to all use time servers so that the times in the error logs on all systems would match. This does not seem to be as easy as I remembered it being for XP systems, and super easy for the Linux systems except that we've lobotomized their ethernet interfaces so that the whole LAN can be taken off of the WAN whenever we need. Consequently, I never got to a working answer with either system. (Update: See final solution at the end of the post.) Here is what I've learned so far:

Network Time Protocol (NTP) works by having branch systems poll systems higher up the tree rather than going to a central hub. Presumably, the base of the tree is a reliable server. Reliability at the branch level is increased by having them poll more than one other system, in fact any number which are then voted against each other. The branching levels are called STRATUMs, and it's bad form to connect to a STRATUM1 system but STRATUM2 or lower is fine.

Astonishingly, in Windows it seems to be set up to use an NTP time server, but to configure it oddly requires nightmarish direct registry modification. Per the following links:

http://www.atomic-clock.galleon.eu.com/support/windows-xp-ntp-time-server.html
http://support.microsoft.com/kb/314054

This set of slides from NIST hints that is could be possible to set everything up by using net time, and then stopping and starting w32time, but that can't be true otherwise M$ and others would have that instead of the registry modifications in their hints.

http://tf.nist.gov/service/pdf/win2000xp.pdf


After reading these links, my thoughts were more along the lines of "maybe the laptops are already set up to time servers and I just need to prove it to myself.". So the question of how to check if a server has been set up led to further dead ends until I decided to give that up too. Apparently, net time and w32tm can be used to find this out, except that not so much on XP. Apparently the /querysntp option doesn't exist for net time on XP, and w32tm fails if the local domain, protocols, or firewall aren't set up right.

http://superuser.com/questions/425233/how-can-i-check-a-systems-current-ntp-configuration

When I tried net time on the laptops I got "System error 5 has occurred. Access is denied." (It worked fine on my own laptop.) This apparently means access to the remote system, apparently if any systems in the local domain are mismatched this will fail, or if there's a firewall issue, or oddly if the computer times are mismatched (hope this isn't it):

http://support.microsoft.com/kb/555644

From a rootkit vendor:

http://www.personalcomputerfixes.com/general-errors/how-to-fix-system-error-5-%E2%80%93-%E2%80%9Caccess-is-denied%E2%80%9D/

And w32tm was full of options, none of which included a simple query as to the setup, as far as I could tell.

So eventually, I decided that the PC times seem really close to my own which I know is on a server, so they must be getting time from a server. How about the linux box?

Finally, something easy:

https://help.ubuntu.com/8.04/serverguide/NTP.html

Getting less eay:

http://www.ntp.org/ntpfaq/NTP-a-faq.htm

Too bad we turned off both the nameservice and even the gateways for these machines because we couldn't be bothered with any sort of sophistication when setting them up to run off of the network. Ah well.

Update: From the first link just above, I learned that it is possible for a sysadmin to run ntpdate directly to do a one-shot clock setup from another server. Presumably I wasn't able to use the gateways on my linux box to get to any other servers, but here's where it all comes together: what about polling the other systems on my LAN that are in the /etc/hosts table? Yes, I was able to reset the time on the linux box by using the windows laptop as my NTP host. Using:

sudo ntpdate [ip of the windows XP system]

and boom the ubuntu time now matches the time on my Windows laptop. Crude but effective, and it wouldn't have worked if my XP firewall had been better set up).

Furthermore, I added an entry in /etc/cron.daily to run ntpdate every day since the unix boxes seem to have an impressive amount of clock drift (a couple of minutes over the course of a month). This required that I add an executable script (named ntpdate per the examples but I think that the name is unimportant) to this directory which contained my ntpdate command. This was covered in the first ubuntu link above, however I also had to do some reading up on cron and found some very nice links for that:

http://www.unixgeeks.org/security/newbie/unix/cron-1.html

http://www.unix.com/unix-dummies-questions-answers/5695-cron-how-tell-s-working.html

http://www.softpanorama.info/Utilities/cron.shtml

The last link above is the one that confirmed that these are indeed just shell scripts in this directory, and ignore lines that begin with the standard shell script comment character #. On a side note, apparently the version of cron that I'm running in ubuntu is called "Nixie cron".

Then I had to correct the timezone. I did this by doing:

sudo dpkg-reconfigure tzdata

Which launches a hilarious terminal server that offers bios-like choices for location that allowed me to pick "New York" and set the time to the correct time zone. I got this from the following link, which is very nice and also reiterates everything that my previous ntp references told me, but in a much prettier way:

https://help.ubuntu.com/community/UbuntuTime