ElasticSearch 5 – How to start using NEST C# .net

In this guide i want to show you how to start in c# .net framework with ElasticSearch version 5, with the library NEST.

WHAT IS ELASTICSEARCH 5

Elasticsearch is a Lucene-based search server with Full Text capability, with support for distributed architectures. All features are natively exposed through RESTful interface, while information is handled as JSON documents.
Is a database that contain many table and date, the table are called “index” and the date are called “documents” is like a no-sql database, as mongodb or couchbase.
In each “index” you can load many “documents” with many different “mapping”.

I suggest to install Kibana to manage ElasticSeach. Starter guide.

CHAPTER MAPPING LINK OFFICIAL DOC.

You can put the mapping on elastic search server throw Kibana dev console.
Mapping is the process of defining how a document, and the fields it contains, are stored and indexed. For instance, use mappings to define:

  • which string fields should be treated as full text fields.
  • which fields contain numbers, dates, or geolocations.
  • whether the values of all fields in the document should be indexed into the catch-all _allfield.
  • the format of date values.
  • custom rules to control the mapping for dynamically added fields.
PUT my_index 
{
  "mappings": {
    "user": { 
      "_all":       { "enabled": false  }, 
      "properties": { 
        "title":    { "type": "text"  }, 
        "name":     { "type": "text"  }, 
        "age":      { "type": "integer" }  
      }
    },
    "blogpost": { 
      "_all":       { "enabled": false  }, 
      "properties": { 
        "title":    { "type": "text"  }, 
        "body":     { "type": "text"  }, 
        "user_id":  {
          "type":   "keyword" 
        },
        "created":  {
          "type":   "date", 
          "format": "strict_date_optional_time||epoch_millis"
        }
      }
    }
  }
}

In the mapping you can define the field, and each field as a datatype. Read the official guide for all datatypes.
Principles datatype:
– boolean
– int | short | long
– date
– text
– keyword
– specialized type (as IP, Completion (autocomplete), Token count datatype)

NEST C#

NEST is a high level client that has the advantage of having mapped all the request and response objects, comes with a strongly typed query DSL that maps 1 to 1 with the Elasticsearch query DSL, and takes advantage of specific .NET features such as covariant results and auto mapping of POCOs. NEST internally uses and still exposes the low level Elasticsearch.Net client.
Link to download NEST.

PRATICAL USE OF NEST C#

#Connect to elastic search server
ElasticClient client = new ElasticClient(new Uri("http://localhost:9200"));

Now we must to create the filter query to take data.
Using Nest to take this is very simple, i show you an example:

#Create our query container
QueryContainer filters = null;
#take a filter on datetime
var filterDate = new QueryContainerDescriptor<Record>().DateRange(t => t.Field("datetimeborn")
.GreaterThanOrEquals(DateTime.MinValue)
.LessThanOrEquals(DateTime.MaxValue));
#Add the filter on query container
filters &= (filterDate);
#You can use the Or
filters |= (filterDate);

Mapping of data type and method:
Boolean => Exists
Date => DateRange
Number => Range
Text Like => WildCard
Text Exact => Term

#Elastic Search Query
POST students/_search
{
  "query":{
    "nested":{
      "path": "information", 
      "query":{
    "wildcard":{"information.name":"*SURNAME*"}
    }
    }
  }
}

#Nest c# query
var filterWildCard = new QueryContainerDescriptor<Record>().WildCard(t => t.Field("information.name").Value("*SURNAME*"));

Now i’have used the Bool Query type, but exist other type, remind you to this official link.

#Execute client search
var response = client.Search<IndexType>(s => s
	.Index("index")
	.Type("mapping")
	.Query(q => q.Bool(bq => bq.Filter(filters))
).From(0).Size(100));

#In the response we find
var documents = response.Documents;

USING NEST SCROLL

The scroll API allows you to efficiently page through a large dataset as it keeps the query alive in the cluster. And if you want all / unlimited data, without pagination.

[CVE-2017-1000253] Linux kernel security bug fixed

How to report on zdnet on their article there is a serious bug with linux kernel identified as CVE-2017-1000253:

§ Centos6 con Kernel < 2.6.32-696.10.3
§ Centos7 con Kernel 7.4 kernels < 3.10.0-693

The bug how report zdnet:

This is a problem with how the Linux kernel loaded Executable and Linkable Format (ELF) executables. If an ELF application was built as Position Independent Executable (PIE), the loader could allow part of that application’s data segment to map over the memory area reserved for its stack. This could cause memory corruption. Then, an otherwise unprivileged local user with access to a Set owner User ID (SUID) or otherwise privileged flawed PIE binary, could gain higher-level user privileges.

The bug was fixed so you must to upgrade your kernel.

I used the guide on the site tecmint.com. It was very helpful and easy, consist to install a new kernel and then switch the boot on the new kernel. Suggest to take a snapshot or a backup of your machine / virtual machine.

 

 

Aruba VPS Centos VMware – Come aumentare lo spazio hdd

Oggi vedremo come aumentare lo spazio (ripartizionare) dell’hard disk, una volta eseguito l’upgrade sul vostro vps aruba.

fdisk -l

parted /dev/sda/

Per creare una nuova partizione per lo spazio aggiunto, lanciare mkpart e rispondere alle varie richieste. Dapprima vi viene chiesto di scegliere il “type” fra” primary” ed “extended” (sceglieremo primary) successivamente viene chiesto il tipo di “File System” (ad esempio scegliamo “ext3” e confermiamo); successivamente viene richiesto l’inizio e la fine della partizione (nel nostro esempio se vogliamo creare una partizione che corrisponda con tutto lo spazio aggiunto, basta indicare come inizio il valore “end” della partizione con il numero più alto e con la dimensione massima per la fine).

A questo punto rilanciando print possiamo verificare che è stata creata la nuova partizione 3.

Ora non resta che creare il Filesystem nella partizione creata: dapprima uscire da “parted” digitando quit e quindi lanciare mke2fs per la partizione creata.

df -h /home/

vgdisplay vg

lvextend -L +2G /dev/mapper/vg-lv_root

resize2fs /dev/mapper/vg-lv_root

Per arrivare a ciò ho seguito due guide, la prima di Aruba e la seconda di linuxtechi, che vi riporto qui di seguito:

http://kb.cloud.it/computing/creare-e-configurare-un-cloud-server/ri-formattare-un-disco-fisso-espanso-linux.aspx
http://www.linuxtechi.com/extend-lvm-partitions/

How to reset mysql root password

mysql

Pratical and best guide that i have tried from many others that founded on web about how to reset mysql root password on linux server.

When you have this error, on you Ubuntu or Centos server (I have tried on Centos, and the user in the original post tried on Ubuntu):

ERROR 1045 (28000): Access denied for user 'root'@'localhost' (using password: NO)

This is the guide to reset your mysql password:

$ sudo -s

# /etc/init.d/mysql stop

# mysqld_safe --skip-grant-tables &

# mysql -u root

 

After logged as user root, reset the password:

mysql> use mysql;

mysql> update user set password=PASSWORD(’__NEW__PASSWORD__’) where User=’root’;

mysql> flush privileges;

mysql> \q

# /etc/init.d/mysql start

 

And then, remember take a backup of your file!!

mysqldump -u root -p --all-databases > alldb.sql

Thank you to http://forum.ubuntu-it.org/viewtopic.php?t=293129

VestaCP upgrade PHP 5 to PHP 7 on CentOS 6

Hi guys,
today i will show to you how to upgrade on VestaCP PHP 5.x to a 2x faster PHP 7!

At start will upgrade EPEL, remember that we use the Centos 6, so EPEL must to be the 6 version.
# wget https://dl.fedoraproject.org/pub/epel/epel-release-latest-6.noarch.rpm
# wget http://rpms.remirepo.net/enterprise/remi-release-6.rpm
# rpm -Uvh remi-release-6.rpm epel-release-latest-6.noarch.rpm

See what PHP version will have:
# yum list installed php*
Installed Packages
php55w.x86_64 5.5.38-1.w6
php55w-cli.x86_64 5.5.38-1.w6

See what PHP version will be available in our package:
# yum list available php* | grep php7
php70w.x86_64 7.0.11-1.w6 webtatic
php70w-bcmath.x86_64 7.0.11-1.w6 webtatic
php70w-cli.x86_64 7.0.11-1.w6 webtatic

remove the actual php 5 version:
# yum remove php*

Install the new php 7 version:
# yum install php70w php70w-cli php70w-common php70w-gd php70w-mbstring php70w-mysql php70w-pdo php70w-xml

Verify that all is good:
# php -v
PHP 7.0.11 (cli) (built: Sep 17 2016 12:52:22) ( NTS )
Copyright (c) 1997-2016 The PHP Group
Zend Engine v3.0.0, Copyright (c) 1998-2016 Zend Technologies

Restart the web server:
# service httpd restart

Thank you to Shay Anderson for his guide.

Day pill – c# divide list into sized group

Hi,
Today i have read on stackoverflow some user that ask how to divide a list into equal part.
So i have decided to write on my blog the code that can help someone.


const int maxSize = 500;
for (int i = 0; i < theList.Count(); i += maxSize) { int takePart = (theList.Count() - i) < maxSize? theList.Count() - i : maxSize; Class[] arrayPart = (from G in theList.Skip(i).Take(maxSize) select new Class {}).toArray(); }

I hope that can help!
Peace and Code!

How to import big sql on Plesk 11 / 12

mysql

Do you need to import a big sql on your plesk and don’t know how to do?
It’s simple and very fast.
Connect throw ssh on your server and access to mysql console:

mysql -uadmin -p`cat /etc/psa/.psa.shadow`

Then is simple, just select the database and execute our sql file.
We must to upload on the server the .sql file that we want to import, if we can optimize the upload process we can upload a zipped sql so then on the server just execute:
unzip ourfile.sql.zip

Now select the database:
use name_database;

Execute the sql file:
source path_to_file.sql

In a few second the big sql is uploaded!

Plesk 12 Centos 6 install OpenDKIM

Plesk opendkim

Today i fought with Plesk 12 and Centos 6 to install OpenDKIM with DOMAIN KEY and SPF.

For domain key and spf is simple, just for each domain on plesk directly active it in the setting mail.

OpenDKIM is a service that must be installed on centos directly.
Let’s start!

Install opendkim and opendkim tools:

yum update
wget -P /tmp http://mirror.pnl.gov/epel/6/i386/epel-release-6-8.noarch.rpm
rpm -Uvh /tmp/epel-release-6-8.noarch.rpm
rm -f /tmp/epel-release-6-8.noarch.rpm
yum install opendkim opendkim-tools

Now we must to create the directory that can be used than with opendkim for keys generation

mkdir -pv /etc/opendkim/keys
chown -Rv opendkim:opendkim /etc/opendkim
chmod go-rwx /etc/opendkim/*

touch /etc/opendkim/KeyTable
touch /etc/opendkim/SigningTable
touch /etc/opendkim/TrustedHosts

Then to simple generate the key and assign the domain to trusted host and signing table, create a simple script.

Create the file:

/opt/generatedkim.sh

and put in it:


#!/bin/bash
# /opt/generatedkim.sh
die () {
echo >&2 "$@"
exit 1
}

[ “$#” -eq 1 ] || die “1 argument required, $# provided, domain required, ex: ./script example.com”

cwd=`pwd`
opendkim=”/etc/opendkim”
location=”$opendkim/keys/$1″
[ -d “$location” ] && die “There is already a directory in the folder, delete folder if you want to create a new one”

mkdir -p “$location”
cd “$location”
opendkim-genkey -d $1 -s mail
chown opendkim:opendkim *
chown opendkim:opendkim “$location”
chmod u=rw,go-rwx *
echo “$1 $1:mail:$location/mail.private” >> “$opendkim/KeyTable”
echo “*@$1 $1” >> “$opendkim/SigningTable”
echo “$1” >> “$opendkim/TrustedHosts”
echo “mail.$1” >> “$opendkim/TrustedHosts”
echo
echo “Put this in the DNS ZONE for domain: $1”
echo
cat “$location/mail.txt”
echo
cd “$cwd”

Now we must use it to generate the domain keys and dns record:


/opt/generatedkim.sh test.de

Put this in the DNS ZONE for domain: test.de

mail._domainkey IN TXT “v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDPzE0GmvFwAQsgcFzopy4zMNWUbL6JM5XIyjBy3bUnANI5axeb
/Lw/GBjUoSFLEiO80Tt8m3A5YrBKcodRQQURYiW6/
YtElhLupHyfcxQhfNLU4z9JUOJKPjcpMZCj0Xv873QgVOl+7U605JdBHSPOx4ybBZwDq68cw9YFYRPmEwIDAQAB” ; —– DKIM key mail for test.de

Create the record dns as the script put out on your domain dns zone.

Remember that if you restart the server you must go up the service of opendkim!!

Thank you to matoski.com

Nodejs 5.5 installed on Plesk 12 with Centos 6

This morning i change my vps on OVH taken other of series VPS Cloud 2016.
I’have installed plesk 12.5 the latest version, but when i try to install nodejs 5.5 on centos it say:

WARNING: C++ compiler too old, need g++ 4.8 or clang++ 3.4 (CXX=g++)

when i try to ./configure it.

I have searched how to install the newer compiler tools on centos

sudo curl http://linuxsoft.cern.ch/cern/scl/slc6-scl.repo > /etc/yum.repos.d/slc6-scl.repo
sudo rpm –import http://ftp.mirrorservice.org/sites/ftp.scientificlinux.org/linux/scientific/51/i386/RPM-GPG-KEYs/RPM-GPG-KEY-cern
sudo yum install -y devtoolset-3

And if you want to utilize it without set environment variables:

scl enable devtoolset-3 bash

And then you can continue with the install of nodejs with the operation of:

wget http://nodejs.org/dist/node-latest.tar.gz
tar zxvf node-latest.tar.gz
cd node-v5.5.0/   //it may change version.
./configure
make
sudo make install

LINQ on Nodejs With node-linq

In this days that i have some free time on my “drawing realtime app” project, i have added a managment of the object with the linq.

In c# i use even the linq, but in javascript this is the first time that i use it with success.
Because i have tested https://linqjs.codeplex.com/ some some some times ago but with big object is obviously slow, is even “client side”.

But now with nodejs i can use the linq “server side”, i have seen some library but in my opinion the best is node-linq created by the wearefractal.
Is fast, the docs are good (but they can insert more).

For example i have used it for extract all the action taken in a “drawing room”, for the date i have used timestamp:

var arrRoom = new LINQ(registerRoomAction).Where(function(roomAction) { return (roomAction.room == socket.room && roomAction.date > singleClient.lastupdate && roomAction.date < now); }).OrderBy(function(roomAction) {return roomAction.date;}).Select(function(roomAction) {return roomAction.data;}).ToArray();

 

So if you want to exract only an item you can use Single():

var singleClient = new LINQ(clients).Where(function(client) { return (client.username == username); }).Select(function(client) {return client;}).Single();

 

It is very fast and simple, if you use linq on c#, you are no problem to use it.

For some question you can contact me, i respond to you.