CS-46X/techreview
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

New draft of tech review for Kirsten.

Browse Source
This commit is contained in:
Danila Fedorin 2019-12-05 02:22:03 -08:00
parent 8f74990a79
commit a2ffe39033
1 changed files with 278 additions and 117 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

395
techreview.latex
View File

@ -1,4 +1,4 @@
\documentclass[10pt, draftclsnofoot,onecolumn]{IEEEtran} \documentclass[10pt, draftclsnofoot,onecolumn, compsoc]{IEEEtran}
\def\changemargin#1#2{\list{}{\rightmargin#2\leftmargin#1}\item[]} \def\changemargin#1#2{\list{}{\rightmargin#2\leftmargin#1}\item[]}
\let\endchangemargin=\endlist \let\endchangemargin=\endlist
@ -30,6 +30,9 @@
\pagebreak \pagebreak
% TODO: She wanted something with "my role"?
% TODO: She wanted a more specific abstract.
\section{Role} \section{Role}
My role in this project is the development of the smart application which My role in this project is the development of the smart application which
will be used to control the Fenceless Grazing Collars in the field. This will be used to control the Fenceless Grazing Collars in the field. This
@ -45,6 +48,35 @@ and electric stimuli. Additionally, the FGC aims to provide information
about the behavior of the farm animals to their keepers, with the goal about the behavior of the farm animals to their keepers, with the goal
of improving their understanding of the livestock. of improving their understanding of the livestock.
\section{Selection Criteria}
Because there is no ideal technical solution, the tradeoffs
of every possible technical solution must be analyzed through specific
critera. The Fenceless Grazing Collar system is intended for actual
use in the field by non-technical personnel, and therefore has
the following concrete design concerns:
\begin{itemize}
\item \emph{Accessibility:} The system must be configurable and usable
without a significant learning curve. This means, among other things,
that the user interface of the system must be in some way familliar
to the users (farmers).
\item \emph{Cost:} The entire idea of the project is to reduce the amount
of time and resources that is currently spent by farmers on herding
livestock. If the system is not cost-effective, it does not efficiently
replace farmers, and therefore, is not worth it to the users.
\item \emph{Maintainability:} While this metric is not necessarily reflected
externally, it's nonetheless an important consideration. The software
written for this project must be easy to support and maintain in the future.
This is a goal for most long-term projects.
\item \emph{Reliability:} The system must be reliable, since failures may lead
to loss of livestock or property. This means that communication between
the various components of the project must be effective and consistent,
and that bugs occurring in the codebase have a high cost to the users.
\end{itemize}
\section{Responsibilities} \section{Responsibilities}
\subsection{Smart Device Application} \subsection{Smart Device Application}
A major component of this project is the creation A major component of this project is the creation
@ -53,15 +85,23 @@ the settings of the wireless collar devices. This
section describes the components of this application. section describes the components of this application.
\subsubsection{Platform} \subsubsection{Platform}
The Android platform makes up more than half of the market share Either the Android or the iOS platform could be used
in the United States \cite{android-share}, with a majority of the remaining share taken for the smart application. These platforms require entirely
up by Apple iOS. Because iOS requires an Apple device to perform different tooling and codebases, as well as different types
development, and because Android devices are of devices (Apple phones only run iOS, while Android devices
on average more than 50\% cheaper than iPhone devices \cite{iphone-price}, we will only run Android).
use the Android platform for developing the mobile application
associated with this project. From the perspective of \emph{Accessibility}, Android is better
% https://www.businessinsider.com/android-stops-us-market-share-decline-2013-5 suited for this project. This is because the Android platform
% https://www.forbes.com/sites/amitchowdhry/2015/02/03/average-iphone-price-increases-to-687-and-android-decreases-to-254-says-report/#4d9d29a3539e makes up more than half of the market share in the United States \cite{android-share},
with a majority of the remaining share taken up by Apple iOS. Furthermore,
because Android devices are on average 50\% cheaper than Apple devices\cite{iphone-price}, the barrier
to purchasing new technology to accomodate the smart application would
be lower if Android was used. This is also a benefit from the \emph{Cost} perspective.
Because the Android platform has advantages in terms of both the \emph{Accessibility}
and the \emph{Cost} aspect, this is the optimal technology to use for the
FGC project.
\subsubsection{Language} \subsubsection{Language}
There exists a variety of available technologies and techniques suitable There exists a variety of available technologies and techniques suitable
@ -81,24 +121,29 @@ platform \cite{android-kotlin}.
%https://facebook.github.io/react-native/ %https://facebook.github.io/react-native/
%https://www.theverge.com/2017/5/17/15654988/google-jet-brains-kotlin-programming-language-android-development-io-2017 %https://www.theverge.com/2017/5/17/15654988/google-jet-brains-kotlin-programming-language-android-development-io-2017
The use of JavaScript was ruled out due to its weak type system. The \emph{Maintainability} persepctive is most imprtant for decisions
The language does not undergo a process called "type checking", in this area: The chosen language may significantly affect the difficulty
which means that certain bugs that can easily be detected of developing and maintaining the smart application. From this perspective,
in other languages may go unnoticed in a JavaScript program JavaScript-based frameworks such as Ionic and React Native are
until they are observed in practice. Since this application at a disadvantage. They have weak type systems, which means that they
must reliably control the wireless collars, this is an do not undergo a process called "type checking". This, in turn,
unnecessary risk. means that certain bugs that can easily be detected in other languages may
go unnoticed in a JavaScript program until they are observed in practice, making
it harder to ensure high software quality. Furthermore, the weak type system
property is also detrimental in terms of \emph{Reliability}: if bugs may
slip directly into the real-world usage of the application, there is
no way to guarantee consistent good behavior.
The choice then remains between Java and Kotlin, both Between Java and Kotlin, Kotlin has a still stronger type system, helping
official languages of the Android platform. For our prevent more bugs at compile-time than Java. It is also a language
purpose, Kotlin has advantages over Java. First, with less "boilerplate" code, which eases maintenance and
Kotlin has a stronger type system, which, as described reduces needless complexity in resulting software. Both
above, helps prevents certain bugs before they are of these are benefits to \emph{Maintainability}. Besides this,
observed in practice. Second, while Kotlin has access because Kotlin is a language that runs on the Java Virtual Machine,
to all features of the Android platform that Java does, there are no features of the Android platform that can be done
it has significantly less "boilerplate" code. This will with Java and not with Kotlin. As such, Kotlin is at the very
help reduce the amount of code requiring maintenance, least equally as powerful as Java, and with the benefits outlined
and thus reduce the number of mechanical issues with the final code. above, it should be the language of choice for the project.
\subsubsection{Backwards Compatibility} \subsubsection{Backwards Compatibility}
Because vendors can customize the Android operating system prior Because vendors can customize the Android operating system prior
@ -127,53 +172,116 @@ attempt to leave the prescribed area. Each data point will
be associated with the collar that produced it, such that be associated with the collar that produced it, such that
behaviors of individual animals can be easily analyzed. behaviors of individual animals can be easily analyzed.
\subsubsection{Storage Type} \subsubsection{Storage Server}
An SQL database will be used for storing the data generated Two major categories of storage servers are available at the
by the collars. The technology chosen for this is MariaDB. present time: SQL and NoSQL databases. SQL-based systems
MariaDB \cite{mariadb} was chosen because of its permissive license store data in relational format, turning each piece of
and compatibility with MySQL \cite{mysql}, another SQL dialect that is very data into a collection of related entries in one or more
commonly used in the industry. There exist other technologies tables. On the other hand, NoSQL systems store data in
for storage, such as MongoDB, which do not use the relational arbitrary formats (i.e., as potentially more complex entities
model used by SQL-based databases. The advantage of NoSQL databases than entries in a table).
is the ability to store data in arbitrary formats, without
specifying prior schema. On the other hand, SQL-based
systems (including MySQL and MariaDB) support concurrency,
which is much more useful for our project due to the future
need to scale the fenceless grazing system. Additionally,
we already know the format of our data, making the benefit
of flexible data storage irrelevant to our use case.
% https://mariadb.org/ SQL and NoSQL are largely equivalent in terms of the criteria
% https://www.mysql.com/ presented at the top of this document. However, because
SQL systems enforce a particular format for data (in
contrast to NoSQL systems, which allow data of various,
unpredictable formats), and because the type of data
that will be stored by the FGC project is known ahead
of time, SQL systems offer a slight advantage to
\emph{Maintainability}. This is because "invalid" data
can be detected and reported immediately as it is stored in
the database. External applications that access the database
(such as the API server) may contain bugs pertaining to
data storage, and having an additional layer of verification
in the storage server will help detect and fix those bugs.
Among SQL systems, three are most commonly used: MariaDB
\cite{mariadb}, MySQL\cite{mysql}, and SQLite. SQLite
is a simple SQL-based database system: it doesn't use
a standalone server, unlike MariaDB and MySQL, and
is simply used from inside other applications as a library.
MySQL and MariaDB, on the other hand, are server-based
database systems. Applications needing to store information
into the databases must first establish a connection to
the database server. MariaDB and MySQL are nearly identical
in terms of functionality, although MariaDB has a more
permissive license.
In terms of \emph{Maintainability}, SQLite is at a slight
disadvantage to the rest of the SQL-based systems due to
its restricted feature set. SQLite supports a subset of
the operations supprted by MySQL and MariaDB, and therefore
may be restrictive in the long run. In terms of \emph{Cost},
MySQL is at a disadvantage due to its prohibitive license:
it is non-free software in terms of price, and although
it has a "free" version, this version receives security
updates and fixes at a slower pace. Since MariaDB
is entirely free and open source, but maintains
the features of MySQL, it is best choice for this project.
\subsubsection{Storage Hardware} \subsubsection{Storage Hardware}
Because MariaDB and MySQL require a server machine, a Linux machine Because MariaDB and MySQL require a server machine, such
will be used to host the database. This machine will be a machine must be part of the final design. This
a Raspberry Pi, a small and low-power Linux single-board can be achieved with a standard server computer,
computer. This computer not only has a full-featured a VPS host, or a single-board computer such as a Raspberry Pi \cite{raspi}.
Debian Linux operating system, but also allows for the
addition of hardware components, which will be used
for communicating with collars in the field. Among the
hardware components compatible with the Raspberry Pi \cite{raspi} is a
LoRa shield, which allows the Pi to send and receive
LoRa signals. This is ideal since the server machine will also act as the LoRa
gateway, serving as the liaison between the collars
deployed in the field and the rest of the project.
Alternative implementations include an AVR-based microcontroller The Raspberry Pi is a promising option in terms of both
equipped with ethernet and / or Bluetooth. Such a controller \emph{Cost} and \emph{Maintainability}. It is a small, single-board
would be integrated with a LoRa receiver and transmitter, computer that costs approximately \$30, and is fully
and programmed to send data to some other storage medium, capable of running a Linux distribution such as
potentially a Virtual Private Server (VPS). This Debian (or Raspbian). After the initial cost
implementation was dismissed due to the additional of the purchase, the Raspberry Pi will not consume
complexity introduced by separating the LoRa receiver significant amounts of power, and thus remain very cheap
and transmitter from the storage medium. This leaves in the long run. The advantage to maintainability is
room for issues such as network errors between due to its official support for various additional
LoRa and the SQL machine, which are entirely hardware modules, such as modules for the LoRa
avoided my using a Linux machine (Raspberry Pi) with support protocol used in other parts of the project. By
for external hardware components (LoRa shield). combining the functionality of the LoRa receiver
and the API server, the complexity of the overall
system can be greatly reduced, making it easier
to design and maintain.
% https://www.raspberrypi.org/ A VPS host would have both benefits and drawbacks
in terms of \emph{Maintainability} and \emph{Cost},
although it is superior in terms of \emph{Reliability}.
A Virtual Private Server is a Linux server that is hosted
by another company for a monthly or yearly fee. The costs
of such servers are fairly small: common VPS packages
cost around \$5 per month, which is an adequate price
for such a product. However, unlike the Raspberry Pi,
these costs would accumulate over time, since a VPS
is a subscription-based service. Furthermore, unlike
the Raspberry Pi, the VPS does not support any additional
hardware. Therefore, another device would need to
be constructed to receive data from the Fenceless Grazing
Collars in the field, and forward that data to the VPS
for storage. This increases the complexity of the system,
thereby making software for it more difficult to develop and
maintain. The main advantage of a VPS host is that it
will be guaranteed to remain functional at all times
without the need for additional support, preventing
unexpected outages and therefore increasing reliability.
For this project, a standard server machine is practically
the worst of both worlds. Unlike the Raspberry Pi, which
has support for additional hardware through a standardized
and officially supported interface, a regular server computer
would require additional modification to be able to interface
with LoRa devices in the field (or, just like the VPS host,
it would require an additional dedicated device to receive
and forward LoRa communication). Furthermore, unlike
a VPS host, it will potentially require support (in
events such as power outages or network failures), which
may result in temporary failures, a major downside from
the perspective of \emph{Reliability}.
Between a VPS host and a Raspberry Pi, the latter appears
better suited for the project due the simplicity of integrating
it into the system. No additional hardware or components
would be necessary, reducing the surface area for failures
(thereby benefitting \emph{Reliability}), and the cost
after the initial investment would be minimal, benefitting \emph{Cost}.
As such, a Raspberry Pi should be used for this project.
\subsection{App API Server} \subsection{App API Server}
An API server is needed to allow users of the Android An API server is needed to allow users of the Android
@ -181,44 +289,93 @@ smart application to interact with the fenceless collars
deployed on farm animals. Smartphones do not have deployed on farm animals. Smartphones do not have
hardware that can communicate using the LoRa protocol, hardware that can communicate using the LoRa protocol,
which is used for collars. As such, an intermediate which is used for collars. As such, an intermediate
device, a server that can communicate with both smartphones device - a server that can communicate with both smartphones
and LoRa, must be used. and LoRa - must be used.
\subsubsection{Server Technology} \subsubsection{Server Hardware}
The server will run on the aforementioned Linux machine. It is possible to use the aforementioned Linux storage machine
This is because it is rather expensive, in terms of both as the hardware for the API server. This would have benefits
price and complexity, to introduce two server machines in terms of \emph{Cost} and \emph{Maintainability},
for the individual tasks of SQL data storage and smartphone while having certain drawbacks in terms of \emph{Reliability}.
app support. It would be cheaper to use the existing hardware for the API
server: no further purchases would be necessary besides
the Raspberry Pi, VPS, or other server described
in the sections above. Furthermore, such a configuration
would be maintainable, since it eliminates the complexity
of managing two separate server instances and their communication
with each other. On the other hand, this approach
would couple the health of the API server to the health
of the database server: if one suffers a hardware or critical
software failure, so will the other. This means that
the API/data server will be a major point of failure.
The server will use the Python \cite{python} programming The alternative approach would be to have a separate
language, which is natively supported by the Raspberry Pi, API server. Since no special-purpose hardware would
to host an HTTP(S) server on the local network. The be needed for this task, by the logic in the Storage Hardware
use of Python will allow for rapid development, and give section, a VPS host would tbe most likely candidate for
direct access to the server's LoRa hardware. While other such a server. This would bring in the advantage
languages, such as Go, are also commonly used for API of \emph{Reliability}, but could complicate the system
implementations, they lack the first-class support that making it less \emph{Maintainable}. Furthermore,
Python receives from the Raspberry Pi project. Additionally, since such a configuration would introduce the need
Python is currently at the top of the popularity rankings for communication between the API server and the data
for programming languages, leading to increased access server, this exposes more area for bugs and faults.
to documentation and support.
The Gunicorn \cite{gunicorn} server will be used to expose the Python Because it is cheaper and less complex to set up
application to the Android application clients. While the API server on the same hardware as the data
other web servers, such as Apache and nginx, are very server (the Raspberry Pi), this should be the
commonly deployed in production, Gunicorn has excellent configuration of choice for this project.
support for Python, which will allow the team to
quickly develop the HTTP application.
% https://www.python.org/ \subsubsection{Server Language}
% https://gunicorn.org/ Virtually any language can be used for developing an API.
Python \cite{python} and Rust are some of the viable
options for this project.
Unlike Python, Rust a complicated type system, which is one of its
main selling points. The type system prevents many
classes of bugs that are common in software products,
especially those that use concurrency. It is therefore,
in a way, more \emph{Maintainable} than th other alternatives,
by the same logic that made Koltin more maintainable than
the alternatives for smart application development.
Python is a dynamically typed language, and as such does
not benefit from type checking directly. It thus suffers
in comparison to Rust in terms of maintainability. However,
Python's main advantage for this project is that it is
the official language of the Raspberry Pi, and therefore
can be used for certain hardware manipulation tasks
that are more difficult in other languages. Because
the API server will be tied to LoRa, which is an external
hardware module, using Python for the entire
project will make the end product \emph{more} maintainable,
since there will be no need for additional frameworks,
libraries, or other components to bridge the gap between
the implementation language and the hardware.
Because of its first-class support for hardware manipulation
on the Raspberry Pi, Python is significantly better for
this project than other possible languages.
\subsubsection{Server Software}
Special software is typically used to expose a web application
to the outside world (which is necessary for other components,
such as the smart application, to access the API server). Apache,
nginx, and Gunicorn\cite{gunicorn} are valid options for this purpose.
Gunicorn is superior in terms of \emph{Maintainability} due to
its simplicity. Whereas both Apache and nginx are general-purpose
server software, which have powerful configuration options and
complex execution models, Gunicorn does one thing: host Python
web applications. Since the API server is not intended to be
a complex system, it will not benefit from additional configurability
provided by Apache or nginx, and will therefore reduce the cognitive
load on team members as they work on the project.
\subsubsection{Authentication} \subsubsection{Authentication}
The mobile app will use JSON web token (JWT) JWT and Cookie-based approaches can be used to perform
to allow the mobile application to make multiple requests authentication in the API server.
without having to continuously provide a username
and password to the server. The idea of JWT is that The idea of JWT is that a JSON (JavaScript Object Notation) object, containing
a JSON (JavaScript Object Notation) object, containing
session information (such as the identity of the current user), session information (such as the identity of the current user),
is encrypted with information from the server. The encrypted is encrypted with information from the server. The encrypted
version of this object is then sent to the user, and can be version of this object is then sent to the user, and can be
@ -227,11 +384,14 @@ the token, the user cannot deliberately make changes to it, preventing
security breaches. Once a user logs in, a JWT token will be generated, security breaches. Once a user logs in, a JWT token will be generated,
containing the identity of the user and an expiration date, and containing the identity of the user and an expiration date, and
returned to the app. The app will then use the token for further returned to the app. The app will then use the token for further
requests. requests. This technology is useful in terms of \emph{Maintainablity},
since it is widely supported by most programming languages, and does
not require the use of a browser. Thus, if the project migrates
to other platforms in the future, the technical cost of implementing
authentication on these new platforms will be negligible.
Cookie-based authentication systems were considered as an alternative; Cookie-based authentication systems are commonly used in web applications,
This is commonly used in web applications, serving as a different serving as a different way to avoid authentication on every API or web request. However,
way to avoid authentication on every API or web request. However,
this approach will not generalize well. Using cookie or session-based this approach will not generalize well. Using cookie or session-based
authentication requires more extensive bookkeeping on the client side, which authentication requires more extensive bookkeeping on the client side, which
complicates the implementation of other clients. In the future, it is complicates the implementation of other clients. In the future, it is
@ -241,26 +401,27 @@ requiring the least additional implementation overhead is preferred.
Because of this criterion, JWT-based authentication is better suited Because of this criterion, JWT-based authentication is better suited
for the API server. for the API server.
Because of the above issues with Cookie-based authentication, JWT
should be used for this project due to its benefits to \emph{Maintainability}.
\subsubsection{API} \subsubsection{API}
There are several ways of communicating data to the mobile application There are several ways of communicating data to the mobile application
from the server. Because writing custom code to encode / decode data from the server. Because writing custom code to encode / decode data
send between the android application (client) and the server is sent between the android application (client) and the server is
time consuming and prone to errors, priority is given to existing time consuming and prone to errors, priority is given to existing
encoding / decoding technologies. Of these technologies, Google's encoding / decoding technologies. Of these technologies, Google's
ProtoBuf \cite{protobuf}, XML, and JSON are the most viable. JSON was chosen ProtoBuf \cite{protobuf}, XML, and JSON are the most viable.
for this project because of the ease with which it can be decoded,
% TODO I guess you can talk about protobuf need to regenerate shit
JSON should be used for this project because of the ease with which it can be decoded,
as well as due to its compatibility with the JavaScript ecosystem. as well as due to its compatibility with the JavaScript ecosystem.
Virtually every language (including Kotlin and Python) has support Virtually every language (including Kotlin and Python) has support
for JSON decoding that is well-tested and supported. This makes for JSON decoding that is well-tested and supported. This makes
JSON the most standard choice for an interchange format. JSON the most standard choice for an interchange format. These
benefits give a huge boost to \emph{Maintainability}, since
The aforementioned HTTP server will thus provide a JSON API little additional work needs to be done to handle JSON in most
to the client. During each HTTP requests, the client will modern systems.
include a JWT token, and, once that token is verified, the server
will process the relevant information and return a JSON object
through HTTP containing the resulting data.
% https://developers.google.com/protocol-buffers
\pagebreak \pagebreak
\begin{thebibliography}{99} \begin{thebibliography}{99}