Wine List

gRPC and JSON

It is easy to provide JSON implementation in gRPC proto file but needs additional effort to setup Envoy proxy that can provide conversion between REST GET/POST/PUT requests and the gRPC:

https://blog.envoyproxy.io/envoy-and-grpc-web-a-fresh-new-alternative-to-rest-6504ce7eb880

Kubernetes

Hello World

Following examples from https://github.com/DevOps-with-Kubernetes/examples

Starting minikube:

minikube start --vm-driver=none

Example helloworld_pod.yaml

apiVersion: v1
kind: Pod
metadata:
  name: example
spec:
  containers:
    - name: web
      image: nginx
      imagePullPolicy: Never
    - name: centos
      image: centos
      imagePullPolicy: Never
      command: ["bin/sh", "-c", "while : ; do curl http://localhost:80/ sleep 10; done"]

Run this using

kubectl create -f helloworld_pod.yaml
kubectl  get pods
kubectl logs  example -c centos

Cluster Service 

service_pod_1.yaml

apiVersion: v1
kind: ReplicationController
metadata:
  name: nginx-1.12
spec:
  replicas: 2
  selector:
    project: service_clusterip
    service: web
    version: "0.1"
  template:
    metadata:
      name: nginx
      labels:
        project: service_clusterip
        service: web
        version: "0.1"
    spec:
      containers:
      - name: nginx
        image: nginx:1.12.0
        ports:
        - containerPort : 80  

service_pod_2.yaml

apiVersion: v1
kind: ReplicationController
metadata:
  name: nginx-1.13
spec:
  replicas: 2
  selector:
    project: service_clusterip
    service: web
    version: "0.2"
  template:
    metadata:
      name: nginx
      labels:
        project: service_clusterip
        service: web
        version: "0.2"
    spec:
      containers:
      - name: nginx
        image: nginx:1.13.1
        ports:
        - containerPort : 80   

service.yaml

apiVersion: v1
kind: Service
metadata:
  name: nginx-service
spec:
  selector:
    project: service_clusterip
    service: web
  ports:
  - protocol: TCP
    port : 80
    targetPort: 80
    name: http 

Run Service

kubectl create -f service_pod_1.yaml
kubectl create -f service_pod_2.yaml
kubectl create -f service.yaml
kubectl get service

kubectl get pods

kubectl describe pods nginx-1.13-2r9rf

kubectl get endpoints 

checker.yaml

apiVersion: v1
kind: Pod
metadata:
  name: clusterip-chk
spec:
  containers:
  - name: centos
    image: centos
    command: ["/bin/sh", "-c", "while : ;do curl http://${NGINX_SERVICE_SERVICE_HOST}:80/; sleep 10; done"] 

Find the logs

kubectl logs -f clusterip-chk

Working with MPI

Message Passing Interface is great because it provides interoperability among different programming platforms (C++, C, and Fortran). There is also support provided for Java, Perl, and Python. Working with large scale Fortran and C++ applications, it can be bit slow to use Oracle for interoperability.

I was working towards having an example working that talks between Fortran and C++.

Downloaded and installed MPI to linux machine:

http://www.mpich.org/downloads/

Found a nice example

http://stackoverflow.com/questions/11944356/send-mpi-message-from-a-c-code-to-fortran-90-code

C++ (Notice, fixed a simple typo from float to double)

# include 
# include 
# include 
using namespace std;

void printarray (double arg[], int length) {
   for (int n=0; n<length; n++)
    cout << arg[n] << " ";
  cout << "\n";
}

int main(int argc, char *argv[] ){
   double a[10];
   int myrank,i;
   MPI::Init ( argc, argv );
   myrank=MPI::COMM_WORLD.Get_rank();
   cout << "rank "<<myrank<<" is c++ rank."<<std::endl;
   for (i=0;i<10;i++){
      a[i]=10.0;
   }
   printarray(a,10);
   MPI::COMM_WORLD.Send(&a[0],1,MPI::DOUBLE_PRECISION,1,100);
   MPI::Finalize();
}

Fortran (Notice that there must be 7 spaces before writing line and if line is too long error use &. For this to work, the file has to be named as .f95

program main
implicit none
include "mpif.h"
integer:: ierr,stat(MPI_STATUS_SIZE)
real(8):: a(10)

call mpi_init(ierr)
a=0
print*,a
call mpi_recv(a(1),10,MPI_DOUBLE_PRECISION,0,100, &

MPI_COMM_WORLD,stat,ierr)
print*,a
call mpi_finalize(ierr)
end program

Compiled C++
mpic++ harmeet.cpp -o harmeet_cpp

Compiled Fortran
mpifort harmeet.f95 -o harmeet_f95

Run them both together
mpirun -n 1 ./harmeet_cpp : -n 1 harmeet_95

Notice there is a : between, otherwise we would get stuck at harmeet_cpp sending message that harmeet_95 never receives.

Plotting Matlab Figures and linking them to Latex

I am writing my thesis which means I need to do lots of latex! I am plotting figures in matlab but it is bit tricky linking matlab figures to latex!

So, here's the perfect way to get things done!

1. Plot all the data
2. Call tightfig script to remove margins (
http://www.mathworks.com/matlabcentral/fileexchange/34055)
3. Save the file in pdf
4. Plot in latex!

Example Latex:
\begin{figure}[!ht]

 \centering

 \includegraphics[scale=0.45]{Figures/Chap2/Fault_Profile.pdf}

 \caption{Different Fault Resistances.}

 \label{fig:fault_profile}

\end{figure}

Example Matlab Script:
%impedance from substation
            impedance=[3.781923798    5.435882686    6.368358163    7.200380563    9.856371439 9.856371439    15.87650559    ...
                16.51528666    16.70110783    17.13156594    17.32078243    18.51808337    21.08402126];
           
           
            voltage_data=[25.9532622445019,25.7653016067929,25.6691255616736,25.5853983811606,25.3569910635690,25.3569868192443,24.9617555108483,...
                24.8825781473615,24.8598716415630,24.8123234278547,24.7888762649503,24.6898624381824,24.6335180770592
               
                26.0814492943857,25.9799517244703,25.9265392586507,25.8813822264662,25.7550839588058,25.7550599002203,25.5310154073116,...
                25.4821039369688,25.4685035318285,25.4401241743564,25.4258036302699,25.3646712093259,25.3299030962158
               
                26.1764298329928,26.1514908385483,26.1386241507187,26.1273111091974,26.0969883677681,26.0969850365599,26.0397214690784,...
                    26.0234564651756,26.0187420587268,26.0089169654977,26.0040301248486,25.9831659832180,25.9712770752272
                ];
           
            fig=plot(impedance',voltage_data(1,:)','kv','markersize',10);
            hold on;
            plot(impedance',voltage_data(2,:)','b*','markersize',10);
            plot(impedance',voltage_data(3,:)','ms','markersize',10);
           
            plot(xlim',ones(1,2)*25,'-.','color','black');
            text(12,25,'25 kV','background','w');
           
           
            plot(xlim',ones(1,2)*25*1.058,'-.','color','black');
            plot(xlim',ones(1,2)*25*0.95,'-.','color','black');
            plot(xlim',ones(1,2)*25*1.05,'-.','color','black');
            plot(xlim',ones(1,2)*25*0.975,'-.','color','black');
            text(11,25*1.056,'Range B Upper Limit','background','w')
            text(11,25*0.95,'Range B Lower Limit','background','w')
            text(11,25*1.05,'Range A Upper Limit','background','w')
            text(11,25*0.975,'Range A Lower Limit','background','w')
           

            xlabel ('Impedance from substation (Ohms)')
            ylabel ('Voltage (kV)')
           
            title ('System voltage profile for different system loadings');
            legend ('100%','60%','20%','Location','SouthWest');
           
            hold off;
            tightfig;
            saveas(fig,'C:\Users\hcheem2\Dropbox\Thesis\Thesis_hcheema\Figures\Chap2\Voltage_Profile.pdf','pdf');

Beware of matlab pu measurements for power systems

Note that Vbase and Ibase are defined to be peak voltage and current values as opposed to the RMS values.

http://www.mathworks.com/help/physmod/powersys/ref/threephasevimeasurement.html

This will result in the measurements to be sqrt(2) times smaller than actual values.

Simplest matlab commands

Want to write that exp(j*t) into cos and sines?

Use the command simple

http://www.mathworks.com/help/symbolic/simple.html

What to see the results in decimals instead of fractions?

Use vpa(ans,4)

What evaluate a function with symbolic expressions?

Use subs(f)

Want to see pretty picture of your complicated equation?

type mupad

Nested symbolic sums?

Go to the mupad and type for example,
sum(sum(sum(1,i=1..j),j=n..N),n=1..N)


State Space modelling?

Step Response - http://www.mathworks.com/help/control/ref/initial.html

Impulse Response - http://www.mathworks.com/help/control/ref/impulse.html

e.g.
a = [-0.5572 -0.7814;0.7814  0];
b = [1 -1;0 2];
c = [1.9691  6.4493];
sys = ss(a,b,c,0);
impulse(sys)

Initial-
http://www.mathworks.com/help/control/ref/initial.html

Wind Energy Readings

Principles of Doubly-Fed Induction Generators (DFIG)
http://www.labvolt.com/downloads/download/86376_F0.pdf

Integration of Distributed Generation in the Power System by Math H.J. Bollen and Fainan Hassan

Simple maximizing problem using GAMS 2

*GAMS three bus Optimization


SETS
N buses
/N1*N3/

ALIAS(N,NP)

*Generator data lamda
table B(N,NP)
N1 N2 N3
N1 -10 6 0
N2 6 -20 4
N3 0 4 -15

TABLE GENDATA(N,*)
PMIN PMAX LAMDA
N1 0 100 5.7
N2 0 100 6.7

TABLE LOADDATA(N,*)
PMIN PMAX LAMDA
N2 0 80 10
N3 0 60 8


VARIABLES
sw
gen(N)
load(N)
delta(N)

EQUATIONS
SoficalWelare
LoadBal(N) ;

*the objective functions
SoficalWelare.. sw =e= SUM(N,LOADDATA(N,'LAMDA')*load(N))-SUM(N,GENDATA(N,'LAMDA')*gen(N));

*load balance
LoadBal(N).. gen(N)-load(N) =e= 100*SUM(NP,B(N,NP)*(delta(N)-delta(NP)));


delta.fx('N1')=0;

gen.lo(N) = GENDATA(N,'PMIN');
gen.up(N) = GENDATA(N,'PMAX');

load.lo(N) = LOADDATA(N,'PMIN');
load.up(N) = LOADDATA(N,'PMAX');

gen.fx('N3')=0;

load.fx('N1')=0;

MODEL ed /SoficalWelare, LoadBal/;
SOLVE ed USING LP MAXIMIZING sw;


DISPLAY gen.l, delta.l,load.l;
DISPLAY LoadBal.m;

Simple maximizing problem using GAMS

*This example uses GAMS to maximize
*y=1+0.5*x+0.25x^2
*y less than x+5
*It uses tables to refer coefficients in the equation

SETS
N index of buses
/N1*N3/ ;

TABLE QUAD(N,*)
coeff
N1 1
N2 0.5
N3 0.25 ;

TABLE LINE(N,*)
coeff
N1 5
N2 1
N3 0 ;

VARIABLES
x
y;

EQUATIONS
ZEq
CEq;

ZEq.. y=e=x*x*QUAD('N3','coeff')+x*QUAD('N2','coeff')+QUAD('N1','coeff') ;
CEq.. y=l=x*x*LINE('N3','coeff')+x*LINE('N2','coeff')+LINE('N1','coeff') ;

MODEL ed /ZEq,CEq/;
SOLVE ed USING NLP MAXIMIZING y;

DISPLAY x.l;
DISPLAY y.l;

Some auto.arima magic

Red is arima based returns, they are not as good because portfolio was not good.
So, I have ten sample stocks. I used auto.arima to pick stocks to invest in for 30 days and stored my returns in dailyReturn


##########################
# loads the data ##
##########################

library(fImport)
stockNames <- c("AAPL", "BAC", "INTC", "GOOG","DIS","DB", "EBK","BHP", "POT", "RIO")
numStock <- length(stockNames)

#adjusted columns indices
adjustedIndex=NULL
for(i in 1:length(stockNames)) adjustedIndex[i]=i*6

#downloads all the stock data
stockData=yahooSeries(stockNames, from="2007-01-01",to="2011-06-30") [,adjustedIndex]


returnsData=returns(stockData)

#
# forecast the data
#

library(forecast)
#goes through each day
dailyReturn=c()
for (i in 1103:length(returnsData[,1]))
{
dailyReturn[i-1102]=0
#goes through each return data
for(returnIndex in 1:length(returnsData[1,]))
{
fr=forecast(auto.arima(returnsData[1:(i-1),returnIndex]))[[4]][[1]]
dailyReturn[i-1102]=dailyReturn[i-1102]+max(fr,0);
}
}


#
# compares again the index
#
indexRr = c(returns(yahooSeries("^IXIC",from="2007-01-01",to="2011-06-30",frequency="d"))[,6])
indexLastMonth=indexRr[1103:length(indexRr)]
save(file='stockdata.rda',stockData,returnsData,dailyReturn,indexLastMonth)
plot(indexLastMonth)
lines(indexLastMonth)
points(dailyReturn,col=2)
lines(dailyReturn,col=2)

Stocks Analysis - PART I

#imports the library
#install.packages("fImport") #installs if required
library(fImport)

#stock names
stockNames=c("GOOG")

#adjusted columns indices
adjustedIndex=NULL
for(i in 1:length(stockNames)) adjustedIndex[i]=i*6

#downloads all the stock data
stockData=yahooSeries(

stockNames, from="2007-01-01",to="2011-06-30") [,adjustedIndex]

#gets the returns
returnData= returns(stockData)

#plots the stock data for all stocks
plot(stockData,main="Prices")

#plots the returns for all stocks
plot(returnData,main="Returns")


#stores the returns data without time series
returnsArray = array(1:1,dim=c(length(returnData[,1]),4))
for(i in 1:length(stockNames)) returnsArray[1:length(returnData[,i]),i]=returnData[,i]


#for plotting the prices data without time series
prices = array(1:1,dim=c(length(stockData[,1]),4))
for(i in 1:length(stockNames)) prices[1:length(stockData[,i]),i]=stockData[,i]

#makes the multiple plots
par(mfrow=c(length(stockNames),1), pch=length(stockNames))

for(i in 1:length(stockNames)) plot(density(returnsArray[,i]),main=paste(stockNames[i]," returns density"))


#makes the qqnorm plots
for(i in 1:length(stockNames)) qqnorm(returnsArray[,i],main=paste(stockNames[i]," returns qq norm"))

#contains back the plotting window
par(mfrow=c(1,1), pch=1)

R Software plotting stock prices and returns

install.packages("fImport")
library(fImport)
x <- yahooSeries("IBM")
plot(x)
r <- returns(x)
plot(r)

More R:
http://www.r-chart.com/2010/06/stock-analysis-using-r.html

Sequential LSE Algorithm

Approximates k in
y = A.k
http://dl.dropbox.com/u/11071453/SquentialLSE.cs

This was written for ANFIS algorithm [1].

I also used Sequential LSE is a project about determining stock prices from news articles. It was not much successful project but Sequential LSE fit in there perfectly and got me what I was looking for.

Note 1: You need alglib library for this.
Note 2: You will notice MatrixOperations class, this uses alglib to provide simple routines for finding matrix inverse, transposes, multiplications.

[1] J.-S. R. Jang, `` ANFIS: Adaptive-Network-based Fuzzy Inference Systems,'' IEEE Trans. on Systems, Man, and Cybernetics, vol. 23, pp. 665-685, May 1993.

Matlab Sugeno

type
anfisedit

Plotting data in matlab

So, I had corrosion data by locations (latitudes and longitudes). I wanted to produce a graph.

So, I had this data array (showing only first three rows):
-82.993 42.293 12
-82.361 42.985 10.3
-82.993 42.293 7.4


>> x = transpose(data(:,1));
>> y = transpose(data(:,2));
>> z= transpose(data(:,3));
>> x_edge=[floor(min(x)):0.1:ceil(max(x))];
>> y_edge=[floor(min(y)):0.1:ceil(max(y))];
>> [X,Y]=meshgrid(x_edge,y_edge);
>> Z=griddata(x,y,z,X,Y);
>> surf(X,Y,Z);


>> Z=griddata(x,y,z,X,Y);

Neural Networks in Matlab

p is the input data
e.g. we have two inputs:
x1 = 1 2 1
x2 = 2 3 1

t is the target data
e.g. assume that we want to add our two inputs
t = 3 5 2

>> net=newff(minmax(p),[8,1],{'tansig','purelin'},'traingd');
>> net.trainParam.epochs = 3000;
>> [net,tr]=train(net,p,t);
>> sim(net,p)


If more hidden layers needed, use
>> net=newff(minmax(p),[8,5,1],{'tansig','tansig','purelin'},'traingd');
and so on

A warm story for Christmas

A real Good Samaritan

http://www.bbc.co.uk/news/magazine-12043294

Have wonderful holidays!

Graphing and Data Analysis libraries C#.NET

This post has a list of some free graphic softwares for C#.NET


http://www.codeproject.com/KB/miscctrl/quickgraph.aspx

QuickGraphs

http://quickgraph.codeplex.com/Thread/View.aspx?ThreadId=43469

Glee


http://www.graphviz.org/

GraphViz


http://www.alglib.net/

alglib (data processing)

http://zedgraph.org/wiki/index.php?title=Main_Page

ZedGraph is a set of classes, written in C#, for creating 2D line and bar graphs of arbitrary datasets. The classes provide a high degree of flexibility -- almost every aspect of the graph can be user-modified. At the same time, usage of the classes is kept simple by providing default values for all of the graph attributes. The classes include code for choosing appropriate scale ranges and step sizes based on the range of data values being plotted.

http://www.codeplex.com/dnAnalytics

* Linear algebra classes with support for sparse matrices and vectors (with a F# friendly interface).
* Dense and sparse solvers.
* Probability distributions.
* Random number generation (including Mersenne Twister MT19937).
* QR, LU, SVD, and Cholesky decomposition classes.
* Matrix IO classes that read and write matrices form/to Matlab, Matrix Market, and delimited files.
* Complex and “special” math routines.
* Markov Chain Monte Carlo (MCMC) sampler classes.
* Bayesian estimators.
* Descriptive Statistics, Histogram, and Pearson Correlation Coefficient.
* Overload mathematical operators to simplify complex expressions.
* Visual Studio visual debuggers for matrices and vectors
* Runs under Microsoft® Windows and platforms that support Mono.
* Optional support for Intel®Math Kernel Library (Microsoft® Windows and Linux)

http://mathnet.opensourcedotnet.info/About.aspx

Math.NET is a mathematical opensource toolkit written in C# for the Microsoft .Net platform. Math.NET aims to provide a self-contained clean framework for both numerical scientific and symbolic algebraic computations. The project is covered mostly under the MIT/X11 license with some optional packages under the GPL or LGPL.

http://www.codeproject.com/KB/cs/csstatistics.aspx

This is a computational statistics class written in C#. The public methods are described below.

http://csharp-source.net/open-source/charting-and-reporting

WebControl is a free chart component for asp.net AND winforms. WebControl for creating charts, that render as images(png, jpg, gif, etc). Supports:

* Line Charts
* Smooth Line Charts
* Column Charts
* Area Charts
* Scattered Charts
* Stacked Column Charts
* Pie Charts
* Stacked Area Charts

Polling data/performing analysis continuously at regular intervals 2

http://piezoelectrics.blogspot.com/2009/12/polling-dataperforming-analysis.html

Windows Scheduler is another alternative. I have been testing Windows Scheduler now. I have been well satisfied with the Windows Service so far.