CS-46X/techreview
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

New draft of tech review for Kirsten.

Browse Source
This commit is contained in:
Danila Fedorin 2019-12-05 02:22:03 -08:00
parent 8f74990a79
commit a2ffe39033
1 changed files with 278 additions and 117 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

395
techreview.latex
View File

@ -1,4 +1,4 @@
\documentclass[10pt, draftclsnofoot,onecolumn]{IEEEtran}
\documentclass[10pt, draftclsnofoot,onecolumn, compsoc]{IEEEtran}
\def\changemargin#1#2{\list{}{\rightmargin#2\leftmargin#1}\item[]}
\let\endchangemargin=\endlist
@ -30,6 +30,9 @@
\pagebreak
% TODO: She wanted something with "my role"?
% TODO: She wanted a more specific abstract.
\section{Role}
My role in this project is the development of the smart application which
will be used to control the Fenceless Grazing Collars in the field. This
@ -45,6 +48,35 @@ and electric stimuli. Additionally, the FGC aims to provide information
about the behavior of the farm animals to their keepers, with the goal
of improving their understanding of the livestock.
\section{Selection Criteria}
Because there is no ideal technical solution, the tradeoffs
of every possible technical solution must be analyzed through specific
critera. The Fenceless Grazing Collar system is intended for actual
use in the field by non-technical personnel, and therefore has
the following concrete design concerns:
\begin{itemize}
\item \emph{Accessibility:} The system must be configurable and usable
without a significant learning curve. This means, among other things,
that the user interface of the system must be in some way familliar
to the users (farmers).
\item \emph{Cost:} The entire idea of the project is to reduce the amount
of time and resources that is currently spent by farmers on herding
livestock. If the system is not cost-effective, it does not efficiently
replace farmers, and therefore, is not worth it to the users.
\item \emph{Maintainability:} While this metric is not necessarily reflected
externally, it's nonetheless an important consideration. The software
written for this project must be easy to support and maintain in the future.
This is a goal for most long-term projects.
\item \emph{Reliability:} The system must be reliable, since failures may lead
to loss of livestock or property. This means that communication between
the various components of the project must be effective and consistent,
and that bugs occurring in the codebase have a high cost to the users.
\end{itemize}
\section{Responsibilities}
\subsection{Smart Device Application}
A major component of this project is the creation
@ -53,15 +85,23 @@ the settings of the wireless collar devices. This
section describes the components of this application.
\subsubsection{Platform}
The Android platform makes up more than half of the market share
in the United States \cite{android-share}, with a majority of the remaining share taken
up by Apple iOS. Because iOS requires an Apple device to perform
development, and because Android devices are
on average more than 50\% cheaper than iPhone devices \cite{iphone-price}, we will
use the Android platform for developing the mobile application
associated with this project.
% https://www.businessinsider.com/android-stops-us-market-share-decline-2013-5
% https://www.forbes.com/sites/amitchowdhry/2015/02/03/average-iphone-price-increases-to-687-and-android-decreases-to-254-says-report/#4d9d29a3539e
Either the Android or the iOS platform could be used
for the smart application. These platforms require entirely
different tooling and codebases, as well as different types
of devices (Apple phones only run iOS, while Android devices
only run Android).
From the perspective of \emph{Accessibility}, Android is better
suited for this project. This is because the Android platform
makes up more than half of the market share in the United States \cite{android-share},
with a majority of the remaining share taken up by Apple iOS. Furthermore,
because Android devices are on average 50\% cheaper than Apple devices\cite{iphone-price}, the barrier
to purchasing new technology to accomodate the smart application would
be lower if Android was used. This is also a benefit from the \emph{Cost} perspective.
Because the Android platform has advantages in terms of both the \emph{Accessibility}
and the \emph{Cost} aspect, this is the optimal technology to use for the
FGC project.
\subsubsection{Language}
There exists a variety of available technologies and techniques suitable
@ -81,24 +121,29 @@ platform \cite{android-kotlin}.
%https://facebook.github.io/react-native/
%https://www.theverge.com/2017/5/17/15654988/google-jet-brains-kotlin-programming-language-android-development-io-2017
The use of JavaScript was ruled out due to its weak type system.
The language does not undergo a process called "type checking",
which means that certain bugs that can easily be detected
in other languages may go unnoticed in a JavaScript program
until they are observed in practice. Since this application
must reliably control the wireless collars, this is an
unnecessary risk.
The \emph{Maintainability} persepctive is most imprtant for decisions
in this area: The chosen language may significantly affect the difficulty
of developing and maintaining the smart application. From this perspective,
JavaScript-based frameworks such as Ionic and React Native are
at a disadvantage. They have weak type systems, which means that they
do not undergo a process called "type checking". This, in turn,
means that certain bugs that can easily be detected in other languages may
go unnoticed in a JavaScript program until they are observed in practice, making
it harder to ensure high software quality. Furthermore, the weak type system
property is also detrimental in terms of \emph{Reliability}: if bugs may
slip directly into the real-world usage of the application, there is
no way to guarantee consistent good behavior.
The choice then remains between Java and Kotlin, both
official languages of the Android platform. For our
purpose, Kotlin has advantages over Java. First,
Kotlin has a stronger type system, which, as described
above, helps prevents certain bugs before they are
observed in practice. Second, while Kotlin has access
to all features of the Android platform that Java does,
it has significantly less "boilerplate" code. This will
help reduce the amount of code requiring maintenance,
and thus reduce the number of mechanical issues with the final code.
Between Java and Kotlin, Kotlin has a still stronger type system, helping
prevent more bugs at compile-time than Java. It is also a language
with less "boilerplate" code, which eases maintenance and
reduces needless complexity in resulting software. Both
of these are benefits to \emph{Maintainability}. Besides this,
because Kotlin is a language that runs on the Java Virtual Machine,
there are no features of the Android platform that can be done
with Java and not with Kotlin. As such, Kotlin is at the very
least equally as powerful as Java, and with the benefits outlined
above, it should be the language of choice for the project.
\subsubsection{Backwards Compatibility}
Because vendors can customize the Android operating system prior
@ -127,53 +172,116 @@ attempt to leave the prescribed area. Each data point will
be associated with the collar that produced it, such that
behaviors of individual animals can be easily analyzed.
\subsubsection{Storage Type}
An SQL database will be used for storing the data generated
by the collars. The technology chosen for this is MariaDB.
MariaDB \cite{mariadb} was chosen because of its permissive license
and compatibility with MySQL \cite{mysql}, another SQL dialect that is very
commonly used in the industry. There exist other technologies
for storage, such as MongoDB, which do not use the relational
model used by SQL-based databases. The advantage of NoSQL databases
is the ability to store data in arbitrary formats, without
specifying prior schema. On the other hand, SQL-based
systems (including MySQL and MariaDB) support concurrency,
which is much more useful for our project due to the future
need to scale the fenceless grazing system. Additionally,
we already know the format of our data, making the benefit
of flexible data storage irrelevant to our use case.
\subsubsection{Storage Server}
Two major categories of storage servers are available at the
present time: SQL and NoSQL databases. SQL-based systems
store data in relational format, turning each piece of
data into a collection of related entries in one or more
tables. On the other hand, NoSQL systems store data in
arbitrary formats (i.e., as potentially more complex entities
than entries in a table).
% https://mariadb.org/
% https://www.mysql.com/
SQL and NoSQL are largely equivalent in terms of the criteria
presented at the top of this document. However, because
SQL systems enforce a particular format for data (in
contrast to NoSQL systems, which allow data of various,
unpredictable formats), and because the type of data
that will be stored by the FGC project is known ahead
of time, SQL systems offer a slight advantage to
\emph{Maintainability}. This is because "invalid" data
can be detected and reported immediately as it is stored in
the database. External applications that access the database
(such as the API server) may contain bugs pertaining to
data storage, and having an additional layer of verification
in the storage server will help detect and fix those bugs.
Among SQL systems, three are most commonly used: MariaDB
\cite{mariadb}, MySQL\cite{mysql}, and SQLite. SQLite
is a simple SQL-based database system: it doesn't use
a standalone server, unlike MariaDB and MySQL, and
is simply used from inside other applications as a library.
MySQL and MariaDB, on the other hand, are server-based
database systems. Applications needing to store information
into the databases must first establish a connection to
the database server. MariaDB and MySQL are nearly identical
in terms of functionality, although MariaDB has a more
permissive license.
In terms of \emph{Maintainability}, SQLite is at a slight
disadvantage to the rest of the SQL-based systems due to
its restricted feature set. SQLite supports a subset of
the operations supprted by MySQL and MariaDB, and therefore
may be restrictive in the long run. In terms of \emph{Cost},
MySQL is at a disadvantage due to its prohibitive license:
it is non-free software in terms of price, and although
it has a "free" version, this version receives security
updates and fixes at a slower pace. Since MariaDB
is entirely free and open source, but maintains
the features of MySQL, it is best choice for this project.
\subsubsection{Storage Hardware}
Because MariaDB and MySQL require a server machine, a Linux machine
will be used to host the database. This machine will be
a Raspberry Pi, a small and low-power Linux single-board
computer. This computer not only has a full-featured
Debian Linux operating system, but also allows for the
addition of hardware components, which will be used
for communicating with collars in the field. Among the
hardware components compatible with the Raspberry Pi \cite{raspi} is a
LoRa shield, which allows the Pi to send and receive
LoRa signals. This is ideal since the server machine will also act as the LoRa
gateway, serving as the liaison between the collars
deployed in the field and the rest of the project.
Because MariaDB and MySQL require a server machine, such
a machine must be part of the final design. This
can be achieved with a standard server computer,
a VPS host, or a single-board computer such as a Raspberry Pi \cite{raspi}.
Alternative implementations include an AVR-based microcontroller
equipped with ethernet and / or Bluetooth. Such a controller
would be integrated with a LoRa receiver and transmitter,
and programmed to send data to some other storage medium,
potentially a Virtual Private Server (VPS). This
implementation was dismissed due to the additional
complexity introduced by separating the LoRa receiver
and transmitter from the storage medium. This leaves
room for issues such as network errors between
LoRa and the SQL machine, which are entirely
avoided my using a Linux machine (Raspberry Pi) with support
for external hardware components (LoRa shield).
The Raspberry Pi is a promising option in terms of both
\emph{Cost} and \emph{Maintainability}. It is a small, single-board
computer that costs approximately \$30, and is fully
capable of running a Linux distribution such as
Debian (or Raspbian). After the initial cost
of the purchase, the Raspberry Pi will not consume
significant amounts of power, and thus remain very cheap
in the long run. The advantage to maintainability is
due to its official support for various additional
hardware modules, such as modules for the LoRa
protocol used in other parts of the project. By
combining the functionality of the LoRa receiver
and the API server, the complexity of the overall
system can be greatly reduced, making it easier
to design and maintain.
% https://www.raspberrypi.org/
A VPS host would have both benefits and drawbacks
in terms of \emph{Maintainability} and \emph{Cost},
although it is superior in terms of \emph{Reliability}.
A Virtual Private Server is a Linux server that is hosted
by another company for a monthly or yearly fee. The costs
of such servers are fairly small: common VPS packages
cost around \$5 per month, which is an adequate price
for such a product. However, unlike the Raspberry Pi,
these costs would accumulate over time, since a VPS
is a subscription-based service. Furthermore, unlike
the Raspberry Pi, the VPS does not support any additional
hardware. Therefore, another device would need to
be constructed to receive data from the Fenceless Grazing
Collars in the field, and forward that data to the VPS
for storage. This increases the complexity of the system,
thereby making software for it more difficult to develop and
maintain. The main advantage of a VPS host is that it
will be guaranteed to remain functional at all times
without the need for additional support, preventing
unexpected outages and therefore increasing reliability.
For this project, a standard server machine is practically
the worst of both worlds. Unlike the Raspberry Pi, which
has support for additional hardware through a standardized
and officially supported interface, a regular server computer
would require additional modification to be able to interface
with LoRa devices in the field (or, just like the VPS host,
it would require an additional dedicated device to receive
and forward LoRa communication). Furthermore, unlike
a VPS host, it will potentially require support (in
events such as power outages or network failures), which
may result in temporary failures, a major downside from
the perspective of \emph{Reliability}.
Between a VPS host and a Raspberry Pi, the latter appears
better suited for the project due the simplicity of integrating
it into the system. No additional hardware or components
would be necessary, reducing the surface area for failures
(thereby benefitting \emph{Reliability}), and the cost
after the initial investment would be minimal, benefitting \emph{Cost}.
As such, a Raspberry Pi should be used for this project.
\subsection{App API Server}
An API server is needed to allow users of the Android
@ -181,44 +289,93 @@ smart application to interact with the fenceless collars
deployed on farm animals. Smartphones do not have
hardware that can communicate using the LoRa protocol,
which is used for collars. As such, an intermediate
device, a server that can communicate with both smartphones
and LoRa, must be used.
device - a server that can communicate with both smartphones
and LoRa - must be used.
\subsubsection{Server Technology}
The server will run on the aforementioned Linux machine.
This is because it is rather expensive, in terms of both
price and complexity, to introduce two server machines
for the individual tasks of SQL data storage and smartphone
app support.
\subsubsection{Server Hardware}
It is possible to use the aforementioned Linux storage machine
as the hardware for the API server. This would have benefits
in terms of \emph{Cost} and \emph{Maintainability},
while having certain drawbacks in terms of \emph{Reliability}.
It would be cheaper to use the existing hardware for the API
server: no further purchases would be necessary besides
the Raspberry Pi, VPS, or other server described
in the sections above. Furthermore, such a configuration
would be maintainable, since it eliminates the complexity
of managing two separate server instances and their communication
with each other. On the other hand, this approach
would couple the health of the API server to the health
of the database server: if one suffers a hardware or critical
software failure, so will the other. This means that
the API/data server will be a major point of failure.
The server will use the Python \cite{python} programming
language, which is natively supported by the Raspberry Pi,
to host an HTTP(S) server on the local network. The
use of Python will allow for rapid development, and give
direct access to the server's LoRa hardware. While other
languages, such as Go, are also commonly used for API
implementations, they lack the first-class support that
Python receives from the Raspberry Pi project. Additionally,
Python is currently at the top of the popularity rankings
for programming languages, leading to increased access
to documentation and support.
The alternative approach would be to have a separate
API server. Since no special-purpose hardware would
be needed for this task, by the logic in the Storage Hardware
section, a VPS host would tbe most likely candidate for
such a server. This would bring in the advantage
of \emph{Reliability}, but could complicate the system
making it less \emph{Maintainable}. Furthermore,
since such a configuration would introduce the need
for communication between the API server and the data
server, this exposes more area for bugs and faults.
The Gunicorn \cite{gunicorn} server will be used to expose the Python
application to the Android application clients. While
other web servers, such as Apache and nginx, are very
commonly deployed in production, Gunicorn has excellent
support for Python, which will allow the team to
quickly develop the HTTP application.
Because it is cheaper and less complex to set up
the API server on the same hardware as the data
server (the Raspberry Pi), this should be the
configuration of choice for this project.
% https://www.python.org/
% https://gunicorn.org/
\subsubsection{Server Language}
Virtually any language can be used for developing an API.
Python \cite{python} and Rust are some of the viable
options for this project.
Unlike Python, Rust a complicated type system, which is one of its
main selling points. The type system prevents many
classes of bugs that are common in software products,
especially those that use concurrency. It is therefore,
in a way, more \emph{Maintainable} than th other alternatives,
by the same logic that made Koltin more maintainable than
the alternatives for smart application development.
Python is a dynamically typed language, and as such does
not benefit from type checking directly. It thus suffers
in comparison to Rust in terms of maintainability. However,
Python's main advantage for this project is that it is
the official language of the Raspberry Pi, and therefore
can be used for certain hardware manipulation tasks
that are more difficult in other languages. Because
the API server will be tied to LoRa, which is an external
hardware module, using Python for the entire
project will make the end product \emph{more} maintainable,
since there will be no need for additional frameworks,
libraries, or other components to bridge the gap between
the implementation language and the hardware.
Because of its first-class support for hardware manipulation
on the Raspberry Pi, Python is significantly better for
this project than other possible languages.
\subsubsection{Server Software}
Special software is typically used to expose a web application
to the outside world (which is necessary for other components,
such as the smart application, to access the API server). Apache,
nginx, and Gunicorn\cite{gunicorn} are valid options for this purpose.
Gunicorn is superior in terms of \emph{Maintainability} due to
its simplicity. Whereas both Apache and nginx are general-purpose
server software, which have powerful configuration options and
complex execution models, Gunicorn does one thing: host Python
web applications. Since the API server is not intended to be
a complex system, it will not benefit from additional configurability
provided by Apache or nginx, and will therefore reduce the cognitive
load on team members as they work on the project.
\subsubsection{Authentication}
The mobile app will use JSON web token (JWT)
to allow the mobile application to make multiple requests
without having to continuously provide a username
and password to the server. The idea of JWT is that
a JSON (JavaScript Object Notation) object, containing
JWT and Cookie-based approaches can be used to perform
authentication in the API server.
The idea of JWT is that a JSON (JavaScript Object Notation) object, containing
session information (such as the identity of the current user),
is encrypted with information from the server. The encrypted
version of this object is then sent to the user, and can be
@ -227,11 +384,14 @@ the token, the user cannot deliberately make changes to it, preventing
security breaches. Once a user logs in, a JWT token will be generated,
containing the identity of the user and an expiration date, and
returned to the app. The app will then use the token for further
requests.
requests. This technology is useful in terms of \emph{Maintainablity},
since it is widely supported by most programming languages, and does
not require the use of a browser. Thus, if the project migrates
to other platforms in the future, the technical cost of implementing
authentication on these new platforms will be negligible.
Cookie-based authentication systems were considered as an alternative;
This is commonly used in web applications, serving as a different
way to avoid authentication on every API or web request. However,
Cookie-based authentication systems are commonly used in web applications,
serving as a different way to avoid authentication on every API or web request. However,
this approach will not generalize well. Using cookie or session-based
authentication requires more extensive bookkeeping on the client side, which
complicates the implementation of other clients. In the future, it is
@ -241,26 +401,27 @@ requiring the least additional implementation overhead is preferred.
Because of this criterion, JWT-based authentication is better suited
for the API server.
Because of the above issues with Cookie-based authentication, JWT
should be used for this project due to its benefits to \emph{Maintainability}.
\subsubsection{API}
There are several ways of communicating data to the mobile application
from the server. Because writing custom code to encode / decode data
send between the android application (client) and the server is
sent between the android application (client) and the server is
time consuming and prone to errors, priority is given to existing
encoding / decoding technologies. Of these technologies, Google's
ProtoBuf \cite{protobuf}, XML, and JSON are the most viable. JSON was chosen
for this project because of the ease with which it can be decoded,
ProtoBuf \cite{protobuf}, XML, and JSON are the most viable.
% TODO I guess you can talk about protobuf need to regenerate shit
JSON should be used for this project because of the ease with which it can be decoded,
as well as due to its compatibility with the JavaScript ecosystem.
Virtually every language (including Kotlin and Python) has support
for JSON decoding that is well-tested and supported. This makes
JSON the most standard choice for an interchange format.
The aforementioned HTTP server will thus provide a JSON API
to the client. During each HTTP requests, the client will
include a JWT token, and, once that token is verified, the server
will process the relevant information and return a JSON object
through HTTP containing the resulting data.
% https://developers.google.com/protocol-buffers
JSON the most standard choice for an interchange format. These
benefits give a huge boost to \emph{Maintainability}, since
little additional work needs to be done to handle JSON in most
modern systems.
\pagebreak
\begin{thebibliography}{99}