FOSSASIA Blog
Author : Ajinkya Ramesh Wavare
Author : Ajinkya Ramesh Wavare
FOSSASIA brings people together to share, create, develop and make things with Open Technologies and software. We organize science hack labs, Open Tech events, meet ups and coding programs with developers, designers and contributors.
Checkout the FOSSASIA Idea’s page for more information.
(ˢᵒᶜⁱᵉᵗʸserver) aims to be a platform for developing collaborative applications.
sTeam server project repository: sTeam.
sTeam-REST API repository: sTeam-REST
sTeam command line lacks the functionality to read and set the object access permissions. The permission bits are: read,write, execute, move, insert, annotate, sanction. The permission function was designed analogous to the getfacl() command in linux. It should display permissions as: rwxmias corresponding to the permission granted on the object.
The the key functions are get_sanction, which returns a list of objects and permissions and sanction_object, which adds a new object and its set of permissions. The permissions is stored as an integer and the function should break the individual bits like getfact().
The permission bits for the sTeam objects are declared in the
access.h
// access.h: The permission bits #define FAIL -1 #define ACCESS_DENIED 0 #define ACCESS_GRANTED 1 #define ACCESS_BLOCKED 2 #define SANCTION_READ 1 #define SANCTION_EXECUTE 2 #define SANCTION_MOVE 4 #define SANCTION_WRITE 8 #define SANCTION_INSERT 16 #define SANCTION_ANNOTATE 32
The get_sanction
method defined in the access.pike
returns a mapping which has the ACL(Access Control List) of all the objects in the sTeam server.
// Returns the sanction mapping of this object, if the caller is privileged
// the pointer will be returned, otherwise a copy.
final mapping
get_sanction()
{
if ( _SECURITY->trust(CALLER) )
return mSanction;
return copy_value(mSanction);
}
The functions gets the permission values which are set for every object in the server.
The sanction_object
method defined in the object.pike
sets the permissions for the new objects.
// Set new permission for an object in the acl. Old permission are overwritten.
int sanction_object(object grp, int permission)
{
ASSERTINFO(_SECURITY->valid_proxy(grp), "Sanction on non-proxy!");
if ( query_sanction(grp) == permission )
return permission; // if permissions are already fine
try_event(EVENT_SANCTION, CALLER, grp, permission);
set_sanction(grp, permission);
run_event(EVENT_SANCTION, CALLER, grp, permission);
return permission;
}
This method makes use of the set_sanction
which sets the permission onthe object. The task ahead is to make use of the above functions and write a sTeam-shell command which would provide the user to easily access and change the permissions for the objects.
The work done during GSOC 2016 by Siddhant and Ajinkya on the sTeam server was merged into the gsoc201-societyserver-devel and gsoc2016-source branches in the societyserver repository.
The merged code can be found at:
https://github.com/societyserver/sTeam/tree/gsoc2016-source
https://github.com/societyserver/sTeam/tree/gsoc2016-societyserver-devel
The merged code needs to be tested before the debian package for the sTeam server is prepared. The testing has resulted into resolving of minor bugs.
The documentation for the sTeam is done using doxygen. The doxygen.pike
is written and used to make the documentation for the sTeam server. The Doxyfile which includes the configuration for generating the sTeam documentation is modified and input files are added. The generated documentation is deployed on the gh-pages in the societyserver/sTeam
repository.
The documentation can be found at:
http://societyserver.github.io/sTeam/files.html
The header files and the constants defined are also included in the sTeam documentation.
sTeam Documentation:
sTeam defined constants:
sTeam Macro Definitions:
Feel free to explore the repository. Suggestions for improvements are welcomed.
Checkout the FOSSASIA Idea’s page for more information on projects supported by FOSSASIA.
FOSSASIA brings people together to share, create, develop and make things with Open Technologies and software. We organize science hack labs, Open Tech events, meet ups and coding programs with developers, designers and contributors.
Checkout the FOSSASIA Idea’s page for more information.
(ˢᵒᶜⁱᵉᵗʸserver) aims to be a platform for developing collaborative applications.
sTeam server project repository: sTeam.
sTeam-REST API repository: sTeam-REST
Peer review is done to ensure that the work performed by one’s peers meets specific criteria. Peer review is used while working in groups because of the belief that one’s peers are able to identify each others errors easily and thus speeding up the time it takes to identify the bugs and get them rectified. The goal of peer review is to ensure that the work satisfies the specifications and identify the aberrations from the standards, help in enhancing, modifying and providing suggestions for improvements. The Review process doesn’t involve management participation.
A lot of work was done in the sTeam server repository during the course of GSoC 2016. There is no support for continuous code integration and testing tools like Code Coverage, Travis CI, Circle CI or Landscape IO as the pike language is not supported by these tools. The project heavily relies on the peer reviews and feedbacks from the members of the community.
The work done by Siddhant on the testing framework and the steam subshell commands was reviewed and tested for bugs. The work which was tested can be found below:
sTeam Server testing framework:
https://github.com/societyserver/sTeam/pull/125
sTeam Server sub shell commands:
https://github.com/societyserver/sTeam/pull/135
The steam project files are segregated into various directories depending upon the tasks performed by them. For an individual to be productively working on the project, the understanding of the directory structure is utmost. The structure needs to be documented. The documentation for the sTeam server was done to facilitate easy understanding of the project structure.
https://github.com/societyserver/sTeam/wiki/sTeam-Server-Structure
Issue’s reported were addressed in individual branches in the fork repositories of the collaborators. The changes made were sent in PR’s to the societyserver-devel and societyserver-source repo. These needed to be merged in the source repository. For resolving the conflicts during merging these, the PR’s were merged into gsoc2016 branch in the societyserver repo.
GSoC 2016 societyserver-devel
// Ajinkya https://github.com/societyserver/sTeam/pull/132 // Siddhant https://github.com/societyserver/sTeam/pull/133
GSoC 2016 societyserver-source
// Ajinkya https://github.com/societyserver/sTeam/pull/138 // Siddhant https://github.com/societyserver/sTeam/pull/137
The merge conflicts between the two PR’s must be resolved and then merged into the main societyserver branches.
Also the issue for setting the permissions for sTeam objects would be addressed in the coming days.
Feel free to explore the repository. Suggestions for improvements are welcomed.
Checkout the FOSSASIA Idea’s page for more information on projects supported by FOSSASIA.
FOSSASIA brings people together to share, create, develop and make things with Open Technologies and software. We organize science hack labs, Open Tech events, meet ups and coding programs with developers, designers and contributors.
Checkout the FOSSASIA Idea’s page for more information.
(ˢᵒᶜⁱᵉᵗʸserver) aims to be a platform for developing collaborative applications.
sTeam server project repository: sTeam.
sTeam-REST API repository: sTeam-REST
The unit testing of the sTeam REST API is done using the karma and the jasmine test runner. The karma and the jasmine test runner are set up in the project repository.
The karma test runner : The main goal for Karma is to bring a productive testing environment to developers. The environment being one where they don’t have to set up loads of configurations, but rather a place where developers can just write the code and get instant feedback from their tests. Because getting quick feedback is what makes you productive and creative.
The jasmine test runner: Jasmine is a behavior-driven development framework for testing JavaScript code. It does not depend on any other JavaScript frameworks. It does not require a DOM. And it has a clean, obvious syntax so that you can easily write tests.
The karma and jasmine test runner were configured for the project and basic tests were ran. The angular js and angular mocks version in the local development repository was different. This had resulted into a new error been incorporated into the project repo. The ‘angular.element.cleanData is not a function’ error is thrown in the local development repository. This error happens when the local version of the angular.js and angular-mocks.js doesn’t match. The testing framework would test you if the versions f the two libraries is not the same.
The jasmine test runner can be accessed from the browser. The karma tests can be performed from the command line.
To access the jasmine test runner from the web browser, go to the url
http://localhost:7000/test/unit/runner.html
To run the karma test suite, run the following command
$ karma start
The unit tests of the sTeam REST service were done using jasmine. The unit tests were written in coffee script. The preprocessor to compile the files from coffee script to javascript is defined in the karma configuration file.
Jasmine Test Runner
First a dummy pass case and a fail case is tested to check there are no errors in the test suite during the test execution.
The localstoragemodule.js which is used in the steam service is injected in the test module. Then the steam service version is tested.
describe 'Check version of sTeam-service', -> it 'should return current version', inject (version) -> expect(version).toEqual('0.1')
steam service should be injected in a global variable as the same service functions shall be tested while performing the remaining tests.
Then the steam service is injected and checked whether it exists or not.
beforeEach inject (_steam_) -> steam= _steam_ describe 'Check sTeam service injection', -> it 'steam service should exist', -> expect(steam).toBeDefined()
The sTeam service has both private and public functions. The private functions cannot be accessed from outside. The private functions defined in the sTeam service arehandle_request
and headers
.
describe 'Check sTeam service functions are defined.', -> describe ' Check the sTeam REST API private functions.', -> it 'steam service handle request function should exist', -> expect(steam.handle_request).toBeDefined() it 'steam service headers function should exist', -> expect(steam.headers).toBeDefined()
The public functions of the sTeam service are then tested.
describe 'Check sTeam service functions are defined.', -> describe ' Check the sTeam REST API public functions.', -> it 'steam service login function should exist', -> expect(steam.login).toBeDefined() it 'steam service loginp function should exist', -> expect(steam.loginp).toBeDefined() it 'steam service logout function should exist', -> expect(steam.logout).toBeDefined() it 'steam service user function should exist', -> expect(steam.user).toBeDefined() it 'steam service get function should exist', -> expect(steam.get).toBeDefined() it 'steam service put function should exist', -> expect(steam.put).toBeDefined() it 'steam service post function should exist', -> expect(steam.post).toBeDefined() it 'steam service delete function should exist', -> expect(steam.delete).toBeDefined()
The test suite written by Siddhant for the sTeam server was tested. The merging of the code from different branches which includes the work done during the course of GSoC 2016 will be merged subsequently.
Karma test runner
Feel free to explore the repository. Suggestions for improvements are welcomed.
Checkout the FOSSASIA Idea’s page for more information on projects supported by FOSSASIA.
FOSSASIA bring’s people together to share, create, develop and make things with Open Technologies and software. We organize science hack labs, Open Tech events, meetups and coding programs with developers, designers and contributors.
Checkout the FOSSASIA Idea’s page for more information.
(ˢᵒᶜⁱᵉᵗʸserver) aims to be a platform for developing collaborative applications.
sTeam server project repository: sTeam.
sTeam-REST API repository: sTeam-REST
sTeam API endpoint testing is done using Frisby. Frisby is a REST API testing framework built on node.js and Jasmine that makes testing API endpoints very easy, speedy and joyous.
Issue. | Github Issue | Github PR |
---|---|---|
sTeam-REST Frisby Test for login | Issue-38 | PR-40 |
sTeam-REST Frisby Tests | Issue-41 | PR-42 |
Write Tests
Frisby tests start with frisby.create with a description of the test followed by one of get, post, put, delete, or head, and ending with toss to generate the resulting jasmine spec test. Frisby has many built-in test helpers like expectStatus to easily test HTTP status codes, expectJSON to test expected JSON keys/values, and expectJSONTypes to test JSON value types, among many others.
// Registration Tests frisby.create('Testing Registration API calls') .post('http://steam.realss.com/scripts/rest.pike?request=register', { email: "ajinkya007.in@gmail.com", fullname: "Ajinkya Wavare", group: "realss", password: "ajinkya", userid: "aj007" }, {json: true}) .expectStatus(200) .expectJSON({ "request-method": "POST", "request": "register", "me": restTest.testMe, "__version": testRegistrationVersion, "__date": testRegistrationDate }) .toss();
The testMe, testRegistrationVersion and testRegistrationDate are the functions written in the rest_spec.js.
The frisby API endpoint tests have been written for testing the user login, accessing the user home directory, user workarea, user container, user document, user created image, groups and subgroups.
The REST API url’s used for testing are described below. A payload consists of the user id and password.
Check if the user can login.
http://steam.realss.com/scripts/rest.pike?request=aj007
Test whether a user workarea exists or not. Here aj workarea has been created by the user.
http://steam.realss.com/scripts/rest.pike?request=aj007/aj
Test whether a user created container exists or not.
http://steam.realss.com/scripts/rest.pike?request=aj007/container
Test whether a user created document exists or not.
http://steam.realss.com/scripts/rest.pike?request=aj007/abc.pike
Test whether a user created image(object of any mime-type) inside a container exists or not.
http://steam.realss.com/scripts/rest.pike?request=aj007/container/Image.jpeg
Test whether a user created document exists or not. The group name and the subgroups can be queried.
eg. GroupName: groups, Subgroup: test.
The subgroup should be appended using “.” to the groupname.
http://steam.realss.com/scripts/rest.pike?request=groups.test
Here “groups” is a Groupname and “gsoc” is a subgroup of it.
http://ngtg.techgrind.asia/scripts/rest.pike?request=groups.gsoc
The unit testing of the sTeam REST API is done using the karma and the jasmine test runner. The karma and the jasmine test runner are set up in the project repository.
The karma test runner : The main goal for Karma is to bring a productive testing environment to developers. The environment being one where they don’t have to set up loads of configurations, but rather a place where developers can just write the code and get instant feedback from their tests. Because getting quick feedback is what makes you productive and creative.
The jasmine test runner: Jasmine is a behavior-driven development framework for testing JavaScript code. It does not depend on any other JavaScript frameworks. It does not require a DOM. And it has a clean, obvious syntax so that you can easily write tests.
The karma and jasmine test runner were configured for the project and basic tests were ran. The angular js and angular mocks version in the local development repository was different. This had resulted into a new error been incorporated into the project repo.
When angular and angular-mocks are not of the same version, these error occurs while running the tests. If the versions of the two javascript libraries don’t match your tests will be testing to you.
The jasmine test runner can be accessed from the browser. The karma tests can be performed from the command line.
Feel free to explore the repository. Suggestions for improvements are welcomed.
Checkout the FOSSASIA Idea’s page for more information on projects supported by FOSSASIA.
FOSSASIA bring’s people together to share, create, develop and make things with Open Technologies and software. We organize science hack labs, Open Tech events, meetups and coding programs with developers, designers and contributors.
Checkout the FOSSASIA Idea’s page for more information.
(ˢᵒᶜⁱᵉᵗʸserver) aims to be a platform for developing collaborative applications.
sTeam server project repository: sTeam.
sTeam-REST API repository: sTeam-REST
Documentation is an important part of software engineering. Types of documentation include:
pike -x extract_autodoc /source
pike -x autodoc_to_html /src /opfile
#!/bin/bash
shopt -s globstar
for filename in ./**/*.pike.xml; do
outputFile=doc/${filename#./}
outputFile=${outputFile%.xml}."html"
if [ -d $(dirname "./"$outputFile) ]; then
touch "./"$outputFile
else
mkdir -p $(dirname "./"$outputFile) && touch "./"$outputFile
fi
pike -x autodoc_to_html $filename "./"$outputFile
done
Autodoc Documentation
The documentation generated by this was less informative and lacked the referrals to other classes and headers. The societyserver project was developed long back but the autodoc utility was introduced in the later versions of pike. As a result the source files lacked the autodoc tags which are required to generate a well informative documentation with bindings to other files.
The sTeam-REST API project made use of the angular-seed to initiate the development during the early phases. However these files still existed in the project. This had lead to a pandemonium and created difficulty in understanding the project. The files had to be removed and the app was in dire need of a restructuring. The following issues have been reported and resolved.
Issue. | Github Issue | Github PR |
---|---|---|
sTeam-REST Issues | Issues | PR |
The new UI can be seen below.
Home
Register
About
The functionality to run the tests using the npm test
command was added to the project. Now the user can run the tests using these commands instead of the traditional approach of running the tests using jasmine-node and specifying the test directory. The domain name of the urls to which the request was sent was changed. The e2e tests and frisby tests were conducted successfully.
e2e Tests.
Frisby Tests
The next step would be to do add more tests for the REST API.
Feel free to explore the repository. Suggestions for improvements are welcomed.
FOSSASIA bring’s people together to share, create, develop and make things with Open Technologies and software. We organize science hack labs, Open Tech events, meetups and coding programs with developers, designers and contributors.
Checkout the FOSSASIA Idea’s page for more information.
(ˢᵒᶜⁱᵉᵗʸserver) aims to be a platform for developing collaborative applications.
sTeam server project repository: sTeam.
sTeam-REST API repository: sTeam-REST
REST is the software architectural style of the World Wide Web. REST (Representational State Transfer) was introduced Roy Fielding in his doctoral dissertation in 2000. Its purpose is to induce performance, scalability, simplicity, modifiability, visibility, portability, and reliability.It has client/server relationship with a uniform interface and is stateless. REST is most commonly associated with HTTP but it is not strictly related to it.
The CRUD(create, retrieve, update and delete ) operations are performed using the HTTP methods.
It is used to retrieve information. GET requests executed any number of times with the same parameters, the results would not change. This makes it idempotent. Partial or conditional requests can be sent. It is a read only type of operation.
Retrieve a list of users:
GET /api.example.com/UserService/users
POST is usually used to create a new entity. It can also be used to update an existing entity. The request will have to do something with the entity provided in the URI.
Create a new user with an ID 2:
POST /api.example.com/UserService/users/2tg
PUT request is always idempotent. Executing the same request any number of times will not change the output. PUT can be used to create or update an existing entity.
Modify the user with an ID of 1:
PUT /api.example.com/UserService/users/1
It is idempotent. PATCH requests only updates the specified fields of an entity.
Modify a user with an id of 1:
PATCH /api.example.com/UserService/users/1
It can be asynchronous or a long-running request. It removes the resource. It can be removed immediately or at a later instance of time.
Delete a user with an ID of 1:
DELETE /api.example.com/UserService/users/1
Installing and activating the REST API
The REST API is developed as an application inside the sTeam server. This simplifies development quite a lot, as we don’t need to restart the server for every change. Instead just the API code gets updated and reloaded. It may eventually be integrated into the core, however the longterm plan is actually to move functionality out of the core, to make development easier.
To get the current version of the API clone the steam-rest repo into your home or to any place where you keep your development repos. Then change to the tools directory of your installation and run import-from-git.
git clone https://github.com/societyserver/steam-rest cd steam-rest git checkout origin/rest.pike export steamrest=`pwd` cd /usr/local/lib/steam/tools ./import-from-git.pike -u root $steamrest /
Note: The new import-from-git.pike script supports importing documents of all mime types.
It is important that the first import is done as root because the API code needs to run with root privileges and it will only do that if the object that holds the source is created as root.
Once the api code is loaded there are just a few tweaks needed to make it work.
We need to fix the mime-type, as the import script is not doing that yet.
OBJ("/sources/rest.pike")->set_attribute("DOC_MIME_TYPE", "source/pike");
Changing the mime type will change the class of the rest api script from Document to DocLpc.
> OBJ("/sources/rest.pike"); (1) Result: 127.0.0.1:1900/rest.pike(#840,/classes/Document,17,source/pike) > OBJ("/sources/rest.pike"); (2) Result: 127.0.0.1:1900/rest.pike+(#840,/classes/DocLpc,529,source/pike,0 Instances, ({ }))
This takes a moment, check the type a few times until it’s done. Then instantiate an object from the source, give it a proper name, and move it to the /scripts/ container”
object rest = OBJ("/sources/rest.pike")->provide_instance(); rest->set_attribute("OBJ_NAME", "rest.pike"); rest->move(OBJ("/scripts/"));
Instantiating the object needs to be done as sTeam-root, in order for it to have permissions to run on behalf of other users.
Once this is done you are ready to start using the API.
The project contains a set of examples and tests for the RESTful API for the sTeam server.
The code is written in coffee script and needs node.js only for coffeescript translation. Deployment can be done as static javascript files, and does not need any kind of dynamic server for the front-end. The back-end is a RESTful API written for the sTeam server as used by steam.realss.com
step 1: install node.js
http://nodejs.org/download/
step 2: clone the repository
git clone https://github.com/societyserver/steam-rest
step 3: install node packages:
npm install
This installs all dependencies (including coffee) for our project into the project’s node_modules directory based on the ‘package.json’ file
step 4: start the server
node_modules/.bin/coffee scripts/server.coffee
but for convenience we can install coffee in the global node environment:
npm install -g coffee-script
so we can just say
coffee scripts/server.coffee
if the server is working you’ll see:
Listening on port 7000
FrisbyJS is used to test the API. It is run through Jasmine and is based on nodejs.
Once you have nodejs installed, run the following statement to install Frisby and Jasmine:
npm install -g jasmine-node frisby
Then execute the test by:
cd project/directory
jasmine-node test/
The karma testing framework is also used for testing the sTeam REST API.
There were some inherent issues with the test framework which were addressed.
Issue. | Github Issue | Github PR |
---|---|---|
Update Readme.md | Update Readme | PR-2 |
Add javascript dependencies | Issue-4 | PR-6 |
Add node dependencies | Issue-5 | PR-7 |
Add angular-mocks.js script for testing the REST services. | Issue-8 | PR-9 |
The project dependencies were not met and this resulted into the error when the project was run on the localhost. The angular-ui-router, angular-bootstrap and bootstrap js frameworks were not installed in the node modules of the project. As a result the bower.json script was modified to include these dependencies.
bower.json
{
"name": "bower",
"version": "0.1",
"private": true,
"ignore": [
"**/.*",
"node_modules",
"bower_components",
"test",
"tests"
],
"dependencies": {
"angular": "",
"angular-route": "~1.4.8",
"angular-ui-router": "",
"angular-bootstrap": "",
"bootstrap": ""
}
}
The node dependencies of karma, frisby and jasmine-node were included in the package.json. These would be installed when the npm install
is executed.
package.json
{
"name": "TechGrind",
"version": "0.1.1",
"private": true,
"dependencies": {
"express": "",
"coffee-script": "",
"morgan": "",
"compression": "",
"method-override": "",
"body-parser": "",
"serve-static": "",
"errorhandler": "",
"bower": "",
"jasmine-node": "",
"frisby": "",
"karma": ""
},
"production_dirs": {
"coffee_src": "src/",
"src": "app/",
"dest": "app_production/"
},
"devDependencies": {
},
"scripts": {
"postinstall": "bower install"
}
}
Feel free to contribute to the repository. Suggestions are welcomed.