commit b23e41ae368be74da013d8c41e4db2c1f1f574f7 Author: Danila Fedorin Date: Tue Mar 17 00:04:35 2020 -0700 Initial draft of the report diff --git a/.gitmodules b/.gitmodules new file mode 100644 index 0000000..cd67bcc --- /dev/null +++ b/.gitmodules @@ -0,0 +1,12 @@ +[submodule "collar"] + path = collar + url = vanilla@dev.danilafe.com:CS-46X/collar.git +[submodule "gateway"] + path = gateway + url = vanilla@dev.danilafe.com:CS-46X/gateway.git +[submodule "server"] + path = server + url = vanilla@dev.danilafe.com:CS-46X/server.git +[submodule "app"] + path = app + url = vanilla@dev.danilafe.com:CS-46X/app.git diff --git a/app b/app new file mode 160000 index 0000000..7b53546 --- /dev/null +++ b/app @@ -0,0 +1 @@ +Subproject commit 7b535461268c942a9c28d23d5da3234c16f0555f diff --git a/collar b/collar new file mode 160000 index 0000000..6cd65f0 --- /dev/null +++ b/collar @@ -0,0 +1 @@ +Subproject commit 6cd65f0a8f4e07bfc76f91517fb2deddacc517c1 diff --git a/collar_list_screen.jpg b/collar_list_screen.jpg new file mode 100644 index 0000000..cedc0d0 Binary files /dev/null and b/collar_list_screen.jpg differ diff --git a/end-of-term-report.latex b/end-of-term-report.latex new file mode 100644 index 0000000..74163f4 --- /dev/null +++ b/end-of-term-report.latex @@ -0,0 +1,457 @@ +\documentclass[10pt, draftclsnofoot,onecolumn, compsoc]{IEEEtran} + +\def\changemargin#1#2{\list{}{\rightmargin#2\leftmargin#1}\item[]} +\let\endchangemargin=\endlist + +\usepackage{subfig} +\usepackage{float} +\usepackage{textcomp} +\usepackage{todonotes} +\usepackage{caption} +\usepackage{pgfgantt} +\usepackage{setspace} +\usepackage{listings} +\linespread{1} + +\def \CapstoneTeamName{Automated Fenceless Grazing} +\def \CapstoneTeamNumber{CS3} +\def \GroupMemberOne{Ryan Alder} +\def \GroupMemberTwo{Danila Fedorin} +\def \GroupMemberThree{Matthew Sessions} +\def \CapstoneProjectName{Automated Fenceless Grazing} +\def \CapstoneSponsorCompany{Oregon State University} +\def \CapstoneSponsorPerson{Bechir Hamdaoui} +\def \DocType{End Of Term Report} + +\newcommand{\NameSigPair}[1]{\par +\makebox[2.75in][r]{#1} \hfil \makebox[3.25in]{\makebox[2.25in]{\hrulefill} \hfill \makebox[.75in]{\hrulefill}} +\par\vspace{-12pt} \textit{\tiny\noindent +\makebox[2.75in]{} \hfil \makebox[3.25in]{\makebox[2.25in][r]{Signature} \hfill \makebox[.75in][r]{Date}}}} + +\begin{document} + +\begin{titlepage} + \pagenumbering{gobble} + \begin{singlespace} + % 4. If you have a logo, use this includegraphics command to put it on the coversheet. + %\includegraphics[height=4cm]{CompanyLogo} + \par\vspace{.2in} + \centering + \scshape{ + \huge CS Capstone \DocType \par + {\large\today}\par + \vspace{.5in} + \textbf{\Huge\CapstoneProjectName}\par + \vfill + {\large Prepared for}\par + \Huge \CapstoneSponsorCompany\par + \vspace{5pt} + {\Large\NameSigPair{\CapstoneSponsorPerson}\par} + {\large Prepared by }\par + Group\CapstoneTeamNumber\par + % 5. comment out the line below this one if you do not wish to name your team + \CapstoneTeamName\par + \vspace{5pt} + {\Large + \NameSigPair{\GroupMemberOne}\par + \NameSigPair{\GroupMemberTwo}\par + \NameSigPair{\GroupMemberThree}\par + } + \vspace{20pt} + } + % \begin{abstract} + % The Fenceless Grazing Collar system aims to reduce the amount of work + % needed by farmers to keep herds of grazing animals. The project + % will be implemented using the LoRa wireless communication protocol to allow + % for long-range interaction between animal-worn collars and a gateway device. + % The gateway device will also provide an HTTP-based JSON API to apply configuration + % changes to collars through an application built for Android mobile devices. + % The MariaDB SQL database management system will be used to store the data + % received from the collar for viewing and analysis. + % \end{abstract} + \end{singlespace} +\end{titlepage} + +\pagebreak +\tableofcontents + +\pagebreak + +\section{Project Recap} +The Fenceless Grazing System project aims to reduce the need for manual labor +for farmers and ranchers caring for large numbers of grazing animals. The +system does so by automating the common task of herding through the use +of GPS-equipped collars that emit negative auditory of electrical stimuli. +Animals that leave a rancher-defined grazing area are met with increasingly +potent uses of the aforementioned stimuli, intended to discourage +such behavior. In addition to the automated herding capabilities, +the FGS also provides data logging capabilities. The data from the +system can be analyzed both by ranchers, in order to make decisions +about the livestock, as well as researchers seeking to measure +the efficacy of the system. + +The Fenceless Grazing System is split into three major components: +\begin{itemize} + \item \emph{GPS Collar Prototype:} This part of the system + is a prototype collar, which will eventually be placed + on a grazing animal. The collar uses a GPS module to + determine its location, and a single-channel LoRa (Long Range) + transmitter to communicate its position and other data + to the \emph{Gateway Server}. + \item \emph{Gateway Server:} This component of the system + is used to manage connections from large numbers of + collars, as well as to store the data transmitted by the + collars. This component is currently implemented + using a Raspberry Pi, a low power, miniature single-board + computer running Linux. The gateway server also provides + a JSON API to external clients, allowing them to view + and modify collar settings. The current reference + client is the \emph{Android Application} + \item \emph{Android Application:} The Android application + uses the JSON API provided by the gateway server to view + the locations of the grazing animals, as well as adjust + the grazing boundaries. To protect the data, the application + uses JSON Web Token authentication. +\end{itemize} + +\section{Current Project State} +During this term, we were able to purchase the majority of the +required hardware, and created a working prototype of the system. +Below are the descriptions of the current state of each +of the project components. + +\subsection{GPS Collar} +The collar controller (implemented as an ATmega328p microprocessor) +is currenly able to interface with the on-board GPS module and +determine its own location, within about a foot. This was tested +in a somewhat populated area, meaning that such precision is in +spite of various obstacles such as buildings and trees +that would not commonly be present in a grazing field. The collar +can also use a single-channel LoRa tranciever to communicate its +location to the gateway server. + +Currently, the collar is assembled using prototyping materials +--- jumper wires and breadboards -- and is not soldered or placed +into a case. This is to allow for quick replacement of broken +or damaged hardware, which occurs somewhat frequently during +development. Despite its current construction, the GPS +collar is quite study, and we anticipate no further need +for soldering or otherwise hardwiring the components. We +do, however, intend to 3D print or laser cut a case. + +The current version of the collar can be seen in Figure \ref{fig:gps_collar}. + +To standardize the communication protocol between the collar and the gateway, +the project uses a data encoding/decoding tool named Protobuf (short for Protocol Buffers). +The tool provides a separate language for describing data structures to be encoded, +and has the advantage of tightly packing data, reducing the number of bytes needed +to store the information. This is especially useful for our purposes, since it +minimzes the amount of data to be transmitted over radio, and therefore minimizes +the potential for error. + +Currently, the communication protocol fits in two Protobuf data structures. The +code is as follows: + +\lstinputlisting{gateway/message.proto} + +The above snippet declares two data structures: a point, which represents +a pair of longitude and latitude, and a ``collar response'' data structure, +which is sent by a collar every time it is prompted by the server. +In the future, the ``collar response'' will also include information +such as the battery level, whether or not the collar is outside of +the prescribed grazing area, and the collar identifier. It is this +information that will be logged by the gateway server. A curious +fact about this message format is that it deliberately uses 32-bit floating +point numbers, rather than their 64-bit version. This is due to the fact +that the ATmega328p microcontroller used by the collar does not +support 64-bit floating point operations. It is unclear at present whether +or not this is an issue. From initial testing, the 16 digits of precision +provided by 32-bit floating point numbers seem sufficient for our purposes. + +\begin{figure}[h] + \center + \includegraphics[width=0.6\linewidth]{gps_collar.jpg} + \caption{GPS Collar Prototype} + \label{fig:gps_collar} +\end{figure} + +\subsection{Gateway Server} +The gateway server can currently correctly serve as the intermediary between +GPS colalrs in the field and the Android application. It does so by providing +two separate services: a REST API, written in Python, and a LoRa client, +written in C. + +The LoRa client is used to interface with the Raspberry Pi LoRa shield, +and provide all the communication-related functionality. Currently, +this entails waiting for location broadcasts from the GPS collars, +decoding them into C data structures from Protobuf, and storing +this information into an SQLite database using SQLite's C API. +Because the C Protobuf API is much more low-level than the +Arduino / ATmega128 API, it requires the users (us) to provide +it with implementation details, such as the method for memory +allocation. We use C's standard \texttt{malloc} function +to do so: + +\lstinputlisting[firstline=5,lastline=15,language=C]{gateway/protobuf.c} + +The REST API uses the Flask microframework to provide an HTTP-based +interface to the Android application. Currently, it receives +login information from the client, verifies it against its +database of user accounts. If the account information is correct, +it creates a JSON Web Token which can then be used by the Android +application in further requests. + +The JSON Web Token is an encoded piece of data in JSON format. +It can contain any arbitrary information, which is inaccessible +to the end-user (the Android application or other client) without +access to the secret key. Thus, while the REST API server can easily +decode JWTs to determine their contents (such as the user ID who +this token belongs to), this information is inaccessible to +external parties, preventing tampering and unauthorized access. +Currently, the REST API provides two operations to the Android +application: a way to get the list of currently active collars, +and an endpoint to list the recent history of a single collar. + +The REST API and the LoRa client share the same SQLite database. +However, wheras the SQLite C API (used by the LoRa client) is +very low-level, the Flask-based API server is capable of using +a high-level interface with the database, called an Object Relational Model (ORM). +Through this interface, the Python code can treat SQL data like Python +objects. This makes it significantly easier to access the modify the data +in the database in a more idiomatic way. However, the ORM interface +is incapable of expressing certain SQL queries. This makes +it necessary for the Flask API to use several SQL queries +to access data, rather than using a single, complex query. +The below code snippet demonstrates our use of the +ORM interface. + +\lstinputlisting[firstline=25,lastline=38,language=Python]{server/fgs/views.py} + +In the above snippet, note, in particular, the various uses +of \texttt{query}. This occurs once in \texttt{Collar.query}, +which retrieves the list of active collars from the database. +Then, for every active collar, the \texttt{query} is used +again, selecting the most recent data point. This means +that the server will have to make one additional SQL +query to the database for every active collar. This +may not scale well in the future, and would necessitate +the use of raw SQL, without the ORM wrapper. + +Additionally, note the use of the \texttt{@jwt\_required} +decorator. Using this decorator, we express that the +endpoint described by the above code (which returns +the list of active collars, together with their +current locations) is only accessible to clients +that have already signed in and were issued a JWT token. +Adding this decorator is sufficient in our code to +protect a route. + +The current version of the collar can be seen in Figure \ref{fig:gateway_server}. + +\begin{figure}[H] + \center + \includegraphics[width=0.3625\linewidth]{gateway_server.jpg} + \caption{GPS Collar Prototype} + \label{fig:gateway_server} +\end{figure} + + +\subsection{Android Application} +The Android application is capable of retrieving login +information from the user, using it to retrieve a JSON Web +Token, and using that token to retrieve and display +the current collar positions on a map. + +When the user initially opens the application, they +are presented with a login screen. The current +appearance of this screen is shown in Figure \ref{fig:login_screen}. + +When the ``Log In'' button is pressed, the application sends +a request to the REST API server with the user name and password +(using secure HTTPS), attempting to authenticate. If the authentication +succeeds, the application stores the generated JWT and uses +it for further interaction with the gateway server. It also +uses the JWT to retrieve a current list of active collars +(triggering the code snippet in the previous section). + +The application is capable of using the OpenStreeMaps API to display +the locations of the active collars on an interactive map. This +map supports zooming in and out and panning. Each individual +collar is presented on the map as a single marker. In +the future, tapping a marker will take the user to a screen +with more information about the selected collar, such +as its recent location history, battery level, and "out-of-bounds" +status. A screenshot of the collar list screen is shown +in Figure \ref{fig:collar_list_screen}. + +\begin{figure}[h] + \center + \subfloat[Login Screen]{ + \includegraphics[width=0.3\linewidth]{login_screen.jpg} + \label{fig:login_screen} + } + \subfloat[Collar List Screen]{ + \includegraphics[width=0.3\linewidth]{collar_list_screen.jpg} + \label{fig:collar_list_screen} + } + \caption{Screenshots from the current version of the Android application.} + \label{fig:screenshots} +\end{figure} + +Currently, the application does not request updates from the server +(this technique is called "polling"). Rather, the user has to manually +press the "back" button and press "log in" again to refresh the +data on the map. Naturally, we intend to change this, so that the map +updates in near real time as the animals move around. + +Though the functionality for regular polling isn't implemented, +the code for handling updated data is present. When receiving a +list of collars, the application must avoid placing duplicate +markers on the map. The application must also remove now-inactive +markers from the map, so that collars that were turned off +are no longer displayed. Additionally, collar names can +change, which must also be reflected in the updated data. +This functionality is implemented by the following Kotlin snippet: + +\pagebreak +\lstinputlisting[firstline=82,lastline=103]{app/app/src/main/java/com/danilafe/fencelessgrazing/CollarListActivity.kt} +In the above code, \texttt{currentSet} is a set of collar IDs that were marked active +in the latest data update. Regardless of whether or not a marker for a collar +with that ID exists (\texttt{collarOverlays[it.id]}) or a new one needs to be +created (\texttt{Marker(map)}), the new collar's ID is added to the set. After +all the updated data is processed, any collars not in the "current" +list are removed from the map. + +\section{To Be Done} +Although it is far along, the project is not yet complete. We believe that the following +functionality / features still need to be completed prior to the code freeze: + +\subsection{Client $\rightarrow$ Collar Communication:} +One of the goals for the Android application is to serve as the main communication channel between +the users of the system (ranchers or farmers) and the collars in the field. +To this end, it must be able to not only view the current locations +of the collars and track their history, but also to modify +the settings on these collars. + +First and foremost, this means that the Android application must be able +to communicate with the collars in some way. The main difficulty in +implementing this is the bridge between the Python-based server application, +which receives API calls from the Android application, and the C-based +LoRa client application, which can actually send and receive data to collars. +We anticipate establishing communication between the two server-based programs +through a message queue in the shared SQL database: whenever the Python server +receives a commands it must relay to the collars, it will place an entry +into the SQL database. The C client will continuously poll the database +for these messages, and send them to collars as they arrive. + +\subsection{Grazing Boundary Tracking and Response} +Currently, while the collars are able to keep track of their location, +they are not able determine whether or not they are in a "valid" location, +or whether they should emit the auditory or electrical stimulus. This +must be implemented, along with the ability to configure the grazing +boundary through the LoRa protocol. + +We will likely define the grazing area to be a 16-point polygon. +The coordinate for each vertex of the polygon will be delivered +through the Protobuf-encoded LoRa communication to the collar, +and an existing algorithm will be used to determine if the current +coordinate of the collar is within thgee polygon. This is likely +sufficient, since grazing fields are unlikely to become large enough +for the Earth's spherical nature to significantly affect the calculations. + +The client had previously stated that it is sufficient for our +collar to shine an LED light to indicate whether or not it is out +of bounds. That is, the actual stimuli are not our main priority, +and thus, we do not have concrete plans to complete that part of +the project. + +\subsection{Multi-channel LoRa Communication and LoRaWAN} +A large subset of the communication needs of our project +has already been implemented by the LoRaWAN protocol, which +is built on top of LoRa. This protocol provides signal scheduling, +allowing for a large number of collar devices to be communicating +without sending data concurrently (and, thus, interfering with +each other). We plan to use LoRaWAN in the final version +of the project, though we are currently using LoRa. + +A major barrier to the use of LoRaWAN for our team +was the need for specialized hardware. While a single-channel +(i.e., capable of only listening to one input at a time) +device is sufficient for LoRa, LoRaWAN requires a +multi-channel (i.e., capable of concurrent communication +with multiple devices) transceiver. We have recently +been able to purchase this component, but have as +yet been unable to integrate it into our system. +Switching to LoRaWAN would also require changes +to the current code of the LoRa client on the +gateway server and the collar. + +\subsection{Data Analysis and Presentation} +Although the current application is capable +of displaying the present location of various +collars, it does not display the grazing +boundary (which is as yet nonexistent +in the code). The application also does not +provide any significant ability to analyze the +gathered data. The client has requested that +address this, especially in preparation +for the engineering expo. + +In particular, the client has asked that +we allow the user to view the history +of an animal's movements on a map, +and be able to determine how many times +the animal has left the prescribed boundary. +This was part of our original design document, +and we plan on including this functionality +in the final product. Other, more visually +interesting diagrams --- such as graphs +of the number of "escapes" per day --- will +be considered, by considered low priority. + +\subsection{Physical Casing} +The team is yet to create a physical casing for the collar +component, which would allow for safer transportation and +more robust testing. This will likely be done with +the help of the university's laser cutter, since +the casing will likely take too long to 3D print +using the university's infrastructure. Additionally, +longer 3D prints have a higher probability of failure, +and may require several attempts. We believe that +using a laser cutter, in combination with standard +CAD software, will be sufficient for us to design +and built a box to protect the collar hardware. + +\section{Problems} +The only problem encountered by the team during +this term is the recent coronavirus outbreak. Under normal +circumstances, our team is organized in a very productive way - +every team member is responsible for a component, and multiple +members can rendezvous to work on integrating the components +with each other. + +Due to the social distancing measures now strongly recommended for +US citizens, we are having a much harder time with integrating +the components, and testing them with each other. Of particular +concern is the integration of the gateway server and the collar +prototype, which need to be in range of each other to be properly +tested. This means that at most one team member can work on these +components at any given time, and switching between team members +becomes especially unproductive. Because the components are +largely physical, it is nearly impossible to test them without +having access to the hardware. This is especially true because +the majority of the bugs that we have encountered were caused +by problems with the physical assembly, and not with the software. + +This issue does not \emph{stop} progress, as much as it \emph{impedes} +it. We are therefore likely going to continue working on the project +with the social distancing measures, having a single team member +work on the hardware components at a time. We have already began +coordinating with Dr. Winters regarding the situation, and we +are currently on track to complete the project despite the delays +caused by the epidemic. + +Other than the recent outbreak, the team has not encountered +any issues. All is well. + +\end{document} diff --git a/gateway b/gateway new file mode 160000 index 0000000..927f3dc --- /dev/null +++ b/gateway @@ -0,0 +1 @@ +Subproject commit 927f3dc6c614fc20ff2300b1d34139deceaea063 diff --git a/gateway_server.jpg b/gateway_server.jpg new file mode 100644 index 0000000..89410c7 Binary files /dev/null and b/gateway_server.jpg differ diff --git a/gps_collar.jpg b/gps_collar.jpg new file mode 100644 index 0000000..1bc5262 Binary files /dev/null and b/gps_collar.jpg differ diff --git a/login_screen.jpg b/login_screen.jpg new file mode 100644 index 0000000..d409f7d Binary files /dev/null and b/login_screen.jpg differ diff --git a/server b/server new file mode 160000 index 0000000..5a20130 --- /dev/null +++ b/server @@ -0,0 +1 @@ +Subproject commit 5a20130c375e42c4841a9f819cafcf28e2fee0d3